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PREFACE AMD ACKNOWLEDGEMENTS 



This report summarizes highlights from an evaluation of the Personnel 
Preparation Program, one of five divisions in the office of Special education 
Programs (OSEP) in the U.S. Department of Education's office of Special Educa- 
tion and Rehabilitative Services (OSBRS). This initial effort was a goal 
evaluation , conducted by a study team from the American Institutes for Research 
(AIR). A separate effort, to be undertaken by AIR in FY87, will be a strategy 
evaluation of one or more aspects of the program. 

The Personnel Preparation Program is the third of five discretionary 
programs to be studied under an OSEP contract with COSMOS Corporation, with 
whoa AIR is participating as subcontractor. The COSMOS project director is 
Robert Yin; the AIR subcontract director is Peggie L. Campeau, who also serves 
as task leader for the Personnel Preparation Program evaluation. 

The other programs being evaluated under this contract are the Handi- 
capped children's Early Education Program, the Media Services/Technology 
Program, the Severely Handicapped Program, and Secondary Education and 
Transitional Services. All five programs operate under the Education of the 
Handicapped Act, as amended. 

OSEP, through this contract, is utilising a program analysis approach 
that assists federal program managers. It takes them through a sequence of 
steps in which they (1) clarify and agree on performance objectives for their 
programs and on strategies for meeting them, (2) make explicit the assumptions 
that are implicit in their choices, and (3) evaluate and improve the plausi- 
bility and efficacy of these strategic choices. 

A particular strength of the approach is that It combines the expertise 
of program managers, a work group of peers and staff, and an external evaluator 
(in this case, AIR), all of whom go through descriptive and analytic processes 
together. The forum for their deliberations is a series of structured work 
group meetings, held once every four to six weeks throughout the evaluation 
process . 
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The work group members for the Personnel Preparation Program goal evalua- 
tion are listed below. They helped to develop some of the study's products, 
and reviewed and critiqued others . Their knowledge of the Personnel Prepara- 
tion Program and its policy context, and the time they invested to make sure 
that this collective effort stayed on track, were essential to the pertinence 
and utility of the goal evaluation process. 
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Director 
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Branch Chief 

Leadership Personnel Branch 
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Jack Tringo 

Related Personnel Branch 
Division of Personnel Preparation 

Marty Kaufman 
Director 

Division of Innovation and Development 

Greg Frane 
Budget Analyst 

Office of Planning, Budget, and Evaluation 
Bill wolf 

acting Branch Chief /Project Officer 
Program Planning and Information Branch 
Division of Program Analysis and Planning 

While the authors alone are responsible for the final product, they would 
also like to thank the work group and other individuals who consented to be 
Interviewed or to provide documents and other information to the study team. 



In particular, we wish to acknowledge the exceptional cooperation of 
project directors and principal investigators of grant projects in the study 
sample, who participated in lengthy telephone interviews with the study team. 



The project was supported by funds from the U.S. Department of Education 
under contract number 300-85>0143. The content of this report does not 
necessarily reflect the views or policies of the U.S. Department of Education, 
nor does mention of trade names, commercial products, or organizations imply 
their endorsement by the U.S. government. 
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EXECUTIVE SUMMARY 



This summary highlights findings and conclusions of a goal evaluation of 
the Personnel Preparation Program, administered by the Division of Personnel 
Preparation (PPP), one of five divisions in the Office of Special Education 
Programs (OSEP) in the U.S. Department of Education's Office of Special Educa- 
tion and Rehabilitative Services (OSERS). 



Overview o f the Personnel Preparation Program 

The program was authorized in 1970 under Part D of the Education of the 
Handicapped Act (P.L. 91-230), although the history of federal involvement in 
the preparation of personnel to work with the handicapped goes back nearly 30 
years . 

The present program, which is the largest of the discretionary programs 
in OSBP, has received total appropriations of over $800 million since 1966 for 
the purpose of increasing the number of fully qualified persons that are 
available to provide education and related services to handicapped children 
and youth. Appropriations exceeded $60 million each year in PY85 and FY86, 
and the authorized funding level for FY87 exceeds $70 million. 

The Personnel Preparation Program awards grants that may be renewed 
annually for up to five years (three years, generally). Grantees may be 
institutions of higher education (XHBs), state education agencies (SEAs), or 
other appropriate nonprofit organizations, who may use their funds in these 
major ways: to develop, improve, and support personnel preparation programs 
(and to provide financial assistance to participants in these r- >grams), to 
develop, evaluate, and disseminate models with broad significance for the 
field of personnel preparation; and to provide technical assistance and 
information to training providers, including parent organizations, so that 
they will be able to meet effectively the needs of children and youth for 
specialized educational and related services, and to interact effectively with 
the system on their behalf. 
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In FY66, 05SRS announced 10 priorities for competition: (1) preparation 
of special educators; (2) preparation of related services personnel; (3) 
parent organisation projects; (4) preparation of personnel to provide special 
education and related services to newborn and Infant handicapped children; (5) 
preparation of leadership personnel; (6) special projects; (7) state education 
agency (SEA) projects; (8) preparation of personnel to work in rural areas; 
(9) preparation of personnel for minority handicapped children; and (10) 
regular educators. Hot ail published priorities need be announced for new 
grant competition each year; for example, the "transition" priority was not 
announced for new grant competition for FY86. 



Overview of the Goal Evaluation Process 

The goal evaluation had three purposes. One purpose was to determine the 
degree to which those strategies the federal program intends to pursue through 
the above major types of grant activities are actually being implemented by 
grantees. The second purpose was to determine, to the extent that data avail- 
able to the study team permitted, if the Personnel Preparation Program Is 
achieving Its objectives. Third, the goal evaluation developed information to 
show if funded activities can logically and plausibly produce the outcomes 
desired by the program, even if actual evidence of these outcomes is insuffic- 
ient. 

The goal evaluation process drew heavily on the assistance of OSEP staff 
and management. Throughout, the task leader met with a work group composed of 
managers and staff representing the program, OSBP, and Office of Planning, 
Budget, and Evaluation (OPBS). They helped to develop some of the study's 
products, and reviewed and critiqued others. Their knowledge of the Personnel 
Preparation Program and its policy context, and the time they invested to make 
sure that this collective effort stayed on track, were essential to the 
pertinence and utility of the goal evaluation process. 

The evaluation approach consists of two parts: a goal evaluation and a 
strategy evaluation. This summary pertains to the goal-oriented phase of the 
evaluation, which is now complete. 
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The main step* in the goal evaluation included: (1) documenting the 
program's logic and underlying assumptions; (2) conducting project reviews of 
a representative sample of 57 projects, with data collection emphasising depth 
in areas important for a program analysis of this type; (3) analysing program 
implementation, performance, and plausibility; and U> drawing conclusions and 
framing recommendations for program management, OSSP, and the work group to 
review in preparation for planning the second, strategy-oriented phase of the 
evaluation. 



Program Objectives and Lo«^ 

The work group reached a consensus on the following statement of the 
Personnel Preparation Program's ultimate goal and objectives: 

HiMmafri »Aai. To enhance education and 
related services for handicapped children 
and youth through the preparation of 
specialised personnel 

"Specialized personnel" means any. personnel, including regular educators, 
who have the knowledge and skills necessary to deliver such services to this 
broad target group, I'sing the word "enhance" deliberately implies that (1) 
fully achieving "free and appropriate public education" for handicapped indi- 
viduals is beyond the direct control or resources of the federal government 
and, in turn, the program and that (2) appropriate roles for the program are 
complementary and catalytic ones. 

To achieve its ultimate goal within these two caveats and those in the 
authorising legislation and regulations, the Personnel Preparation Program 
directs its efforts to three enabling objectives : 

e To produce more qualified personnel to serve children and youth 
who are handicapped 

e To improve the quality of personnel trained to serve children 
and youth who are handicapped 

e To expand the capacity of the system for personnel development 
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The Personnel Preparation Program utilizes eight major strategies to 
attain these three objectives; 
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1. Supporting recruitment and retention 

2. Targeting critical needs 

3. Supporting model program development, evaluation, and dissemin- 
ation 

4. supporting leardership development 

5. Encouraging state and professional standards 

6. Supporting parent organization projects 

7. Building capacity 

8. Promoting Institutionalization 

Figure 1 portrays the logic of the overall program. It shows the rela- 
tionships among events that influence program design, implementation, and 
capacity to meet these objectives. Figure 2 shows the relationship among 
federal strategies, grant activities, and program objectives. The causal 
assumptions implied by the two figures are made explicit in the full report of 
the goal evaluation. 

These major points are relevant to the Figures 1 and 2: 

• The Personnel Preparation Program pursues particular strategies 
through activities that grantees carry out at the state, 
Institutional, and local level. (These strategies and activities 
are the row and column labels, respectively, in Figure 2.) 

e Thus, the grant programs are the primary mechanism for implement- 
ing federal strategies and legislative intent. 

e The matrix conveys the expectation that, in aggregate, (1) 
projects in a particular priority area will contribute more to 
one program objective than to the other two, and that (2) the 
means they implement will be congruent with the federal 
strategy(ies) that are "attached- to that objective. 

e It is possible to focus grant competitions (for selected 

priorities) to accommodate one or more of the strategies (and 
program objectives). 
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POLICY INPUTS 



PROGRAM INPUTS 
(by DPP) 



USING SEVERAL 
FEDERAL STRATEGIES 



THROUGH GRANT 
PROGRAM ACTIVITIES 



TO ACHIEVE 
PROGRAM OBJECTIVES | 



THAT CONTRIBUTE TO THE 
ULTIMATE PROGRAM GOAL 



Congress 

• Enabling legisla- 
tion 

P.L. 98-199 1 Part D 

a Regulations 
14 CFR # Part 318 

a Appropriations 
$61,2*0,000 (FY86) 



Executive Agencies 

a Administration 
policy directives 

a 0MB policy direc- 
tives 

a ED policies 

a OSERS priorities 

The Field (Consti - 
tuencies) 

a Heeds analyses, 
data 

a Priority sugges- 
tions 

a Peer review 



Program support 
Grant program admlnist ration 



a 

a CSPD support 



Leadership and technical 
assistance to the field of 
personnel preparation 
(all levels, as feasible) 

Coordination and collabora- 
tion with other agencies 
regarding pers ,nel pre- 
paration 



(T) Supporting recruit- 
ment and retention 

(?) Targeting critical 
need 8 

Supporting Bodel 
program development, 
evaluation, and 
dissemination 

Supporting leader- 
ship development 

Encouraging state 
and professional 
standards 

(b) Supporting parent 
organization 
projects 

^) Building capacity 

(?) Promoting institu- 
tionalization 



• Program develop- 
ment, improvement, 
and support, in- 
cluding stipends 

a Model program 
development , 
evaluation, and 
dissemination 

• Technical assis- 
tance and infor- 
nat ion 



a Produce no re 
qualified 
personnel to 
serve children 
and youth who 
are handicapped 

a Improve the qual- 
ity of personnel 
trained to serve 
children and youth 
who are handi- 
capped 

a Expand the capac- 
ity of the system 
for personnel 
development 



NOTE: Figure 2 shows the relationship among strategies, 
activities t and objectives. 



a Enhance education 
and related services 
for handicapped 
children and youth 
through the prepara- 
tion of specialised 
personnel 



Figure 1. Personnel Preparation Program Logic Model 
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PROGRAM OBJECTIVES/ 
FEDERAL STRATEGIES 


1 Program Development, 

I Improvement , and Support , 

II Including Stipends 


GRANT ACTIVITIES 

Model Development, 
Evaluation* and 
Dissemination 


Technical Assis- 
tance and 
Information 


Produce no re qualified 
personnel • . . 

(T^ Supporting recruitment and 
retention 

(5) Targeting critical needs 
areas 


1 Special Educators Q © 
1 Kelated Services CD Cb 
1 Rural d) (2 

1 Infnnr 111 f2 

1 in* an t ^ ( __ \Ar 
Transition |^ (3> C& 
Minority 1^ (J) © 






Improve the quality of 
personnel . . . 

(?) Supporting model program 
developrent, evaluation, 
and dissemination 

(£) Supporting leadership 1 
development 1 

Encouraging state and pro- 1 
fessional standards J 


1 Leadership Projects © 


Special Projects ® 

I 




Expand the capacity of I 
the system for personnel I 
development ... I 

Supporting parent organiza- I 
tlon projects I 

(7J Support ine improvements in 1 
system capacity 1 

(?) Promoting institutionalization 1 


"ITT 

**fc^***oa, a*U UkO bU4 Bf X*X 

SEA Projects Q> 




f*» 

Parent Organization 
Projects (£) 

I 
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Figure 2. The Intended Relationship Among Program Objectives, Federal Strategies* 
Grant Activities, and Primary Foci of Competitions (FY86) 



• The essential core of grantmaking activity is represented by 
the five clusters of primary activity depicted in Figure 2 
(see Roman numerals in five cells). 

e Cell entries indicate the main emphasis of FY86 grant 

activity. These clusters might be consituted differently, 
depending upon how each competition area is defined for a 
particular fiscal year. 

The (above) gross classification scheme that Figures 1 and 2 provide 
served two purposes in the goal evaluation. One wis to show the Personnel 
Preparation Program's overall strategic plan, where the federal investment in 
grants is intended to generate the most mileage toward one of the three 
objectives. (The work grovp realised that projects will implement strategies 
in addition to those shown as their primary emphasis in Figure 2.) The second 
purpose of the classification scheme was to provide the conceptual underpin- 
nings for planning data collection and analyses. 



Data Collection Approach and Related Caveats 

The study team carried out 57 confidential project reviews, in which the 
primary data sources were information in grant files and 75-minute (average) 
focused telephone interviews with project directors and/or principal investi- 
gators. (One project was dropped because available information was too minimal 
to include it in subsequent analyses*) 

The study sample consisted of subsamples drawn from each of the program's 
priority areas, shown as cell entries in Figure 2. For the most part, projects 
were drawn at random from FY86 continuations Whose initial year for their 
current grant was FY84. 

Restricting data collection to currently operating projects 9 most of 
which began in FY84, ensured that they had been running long enough to have 
learned lessons from their implementation experience that would be very infor- 
mative for a program analysis of the type conducted in a goal evaluation. 
Also, better cooperation was expected from project staff whose projects were 
currently operating than from projects that had been completed or discontinued. 
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On the other hand, data collection from "live" projects necessarily restricted 
the study to conclusions on p rospective program performance supported by 
evidence that grantees said they were collecting or were likely to present in 
their final performance reports. 



These additional caveats apply to conclusions from the goal evaluation: 

e It is not within the scope of a goal evaluation to collect 
primary data on project accomplishments, or to capture all 
relevant perspectives. Project reviews raly on two major 
secondary data sources: initial and continuation applications 
in grant files, and interviews with project directors or 
principal investigators. Although Interviews were conducted on 
a confidential basis, and most interviewees seemed to be 
candid, it is possible that some relevant information was not 
communicated. 

e Evaluation resources for the goal evaluation did not permit 
data collection from third parties, such as consumers (agencies 
who subsequently utilize the personnel trained and the models 
or programs developed through grant activities). They could 
have indicated the extent to which these products are meeting 
their critical needs and are found to be high-quality, useful, 
and effective. 

e The goal evaluation sample is small in proportion to the size 
of the program, although it is representative of the broad 
array of Personnel Preparation Program grant activities, and 
six of the eleven subsamples constituted between 25% and 37X of 
their sampling pools. 

e Conclusions pertain specifically to federal strategies as the 
Personnel Preparation Program perceived them, and grantees 
implemented them, in grant activities operating in FY86. 

a Goal evaluations do not examine program management procedures 
per se , but do try to determine whether intended major program 
inputs (see Figure 1, Column 2) occur at a level that supports 
program objectives and the federal strategies that are pursued 
to attain objectives. 



Major Conclusions 



The generally positive findings presented in the full report of the goal 
evaluation justify the conclusions that follow, but also indicate areas that 
could profit from further examination in the next phase of the evaluation. 




// 

Strategic Can Be Implemented Through Grant Activities 
To An Extant That Supports Program Objectives 

and 

Project Results Support Program Objectives 

All projects in the study sample were judged to be implementing (l) the 
federal strategies that were expected to be their primary emphasis and, In 
addition, (2) one or more of the strategies associated with other program 
objectives (and competition foci}. 

Overall, the nature of quantitative and qualitative evidence of their 
activities and accomplishments, provided in the full report of the goal 
evaluation, indicates a good fit with program objectives. (See below.) 

K any Project Results Are Well Documented 

Nearly 80% of the study sample claimed to be achieving outcomes that 
pertain to the first program objective, "to produce more qualified personnel." 
They indicated that their supporting data included: numbers of individuals 
recruited, trained, and gradw ed (by level and specialty); number of program 
graduates who subsequently enter careers in special education in roles and 
areas for which they were trained; number and nature of the training, technical 
assistance, and dissemination activities that grantees carried out; and the 
number and nature of the models and materials they developed. 

Over 30% of the study sample reported outcomes claimed to have data 
to support the second program objective, "to improve the quality of personnel 
trained." These data, however, are subjective and qualitative. For example, 
evidence of model quality, improved competence, and use of state-of-the-art 
practice in personnel preparation consisted mostly of subjective assessments of 
"experts," project staffs (who may both design and implement the model during 
its developmental tryout), and participants' instructors or supervisors. 
Although soft, such data served the formative evaluation needs of these model 
and program development projects very well. Moreover, as these three-year 
grant activities are presently focused, it may not be feasible to expect 

xiii- 
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grantees to obtain data that would rigorously support thia federal program 
objective. 

Store than 75% of the study sample reported outcomes that constituted a 
wide variety of system improvements which would support the third program 
objective, "to expand the capacity of the system for personnel development." 
However, much of their corroborating evidence probably will not be provided in 
final performance reports in a form that makes it feasible for federal program 
staff to extract and aggregate. 

Program Logic and Assumpt ions Are Valid 

In the type of analysis characteristic of a goal evaluation, judgments of 
the validity of program logic and assumptions, and the plausibility of program 
objectives, are based on evidence of "congruence,- rather than by testing 
cause-effect linkages. In theory, such an analysis may reveal that what 
projects in the field are actually attempting in their day-to-day operations 
is not consistent with expectations at the federal program level. However, in 
ths Personnel Preparation Program's case, (1) a close correspondence was found 
between expected and reported emphases on federal strategies through major 
kinds of grant activity, and (2) the results and corroborating data that 
grantees in the study sample claim to have will support federal program 
objectives. 

In short, no major incongruities with the logic model are apparent from 
what is actually being attempted through the operating grant projects in the 
study sample. 



Recommendat i>ms 

Thi full report of the goal evaluation presents two types of recommenda- 
tions. One set suggests actions that could be taken immediately to address 
problems or information gap* the goal evaluation identified. A second set 
identifies candidate topics that could be examined in the strategy evaluation 
phase of the study. 
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I. OVERVIEW OF THE PERSONNEL PREPARATION PROGRAM 



For nearly 30 years without interruption, the federal government has 
authorized grants to support the preparation of specialized personnel to edu- 
cate children and youth who are handicapped. The current program is adminis- 
tered by the Division of Personnel Preparation* Office of Special Education 
Programs (OSBP), office of Special Education and Rehabilitative Services 
(OSBRS), U.S. Department of Education. The program was originally authorised 
in 1970 under Part D of the Education of the Handicapped Act (P.L. 91-230). 
Known today as the Personnel Preparation Program, it is the largest discre- 
tionary program in OSBP. 



Since 1966, the Personnel Preparation Program has received total appro- 
priations of over $800 million for the purpose of increasing the number of 
fully qualified persons that are available to provide education and related 
services to handicapped children and youth. Appropriations for the Personnel 
Preparation Program since 1978 are as follows: 



Throughout its history, federal legislation for the development of per- 
sonnel to provide effective services to handicapped children and youth has 
been aimed at improving the quality and increasing the quantity of special 
educators and related services personnel. 



Funding History 



1978 
1979 
1980 
1981 
1982 
1983 
1984 
1985 
1986 
1987 
1988 
1989 



$45,375,000 
55,375,000 
55,375,000 
58,000,000 
49,300,000 
49,300,000 
55,540,000 
61,000,000 
61,250,000 
70,400,000 
74,500,000 
79,000,000 



(authorization) 
(authorization) 
(authorization) 



Legislative History 



o 

ERJ.C 

7- 



-1- 

22 



The history of federal involvament in the preparation of personnel to 
work with the handicapped goes back to 1958, when P.L. 85-926 established 
grants to educate teacher trainers in mental retardation. Legislation during 
the 1960s expanded training grants to Include teachers of all types of 
handicapped children. In the Elementary and Secondary Education Act (ESKA) 
Amendments of 1966 (P.L. 89-570), Congress added a new Title VI creating both 
a program of grants to the states to assist In the education of handicapped 
children and a distinct unit within the Office of Education— the Bureau of 
Education for the Handicapped. 

In 1970, further ESBA amendments— which became known as the Education of 
the Handicapped Act (P.L. 91-230) — consolidated into one act a number of pre- 
viously separate grant authorities relating to handicapped children. Part D 
of this act authorized appropriations for discretionary training grants through 
fiscal year 1973; Congress has subsequently reauthorized these grants on 
several occasions through fiscal year 1987. 

Two additional pieces of legislation in the 1970s brought significant 
changes for the education of the handicapped. The Education Amendments of 
1974 (P.L. 93-380) authorized a six-fold Increase In entitlement (formula) 
funds (from $100 million to $600 million) to assist states in achieving the 
goal of providing full educational opportunities for all handicapped children 
in the public schools. The Education of All Handicapped Children Act of 1975 
(P.L. 94-142), which has become known as a civil rights act for the handicap- 
ped, expanded the provisions of previous legislation with the purpose of 
ensuring a free, appropriate public education for all handicapped children 
between the ages of 3 and 21 by 1980. In order to bring about the integration 
of more handicapped children with nonhandl capped children in the regular 
classroom, the Act required the adequate preparation of regular education 
personnel to meet the needs of handicapped students. 

In response to the passage of P.L. 94-142, BBH (now OS BP) established the 
training *»ff regular educators as another priority area for funding projects 
under the discretionary grants program authorised by Part D of P.L. 91-230. 
P.L. 94-142 did not change Part 0. However, it did expand the state grant 
program authorized by Part B and required states to submit plans for a 
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Comprehensive System of Personnel Development (CSPD) . Under this provision, 
states ere to provide needs-based training for both special educators and 
regular educators to ensure that teachers of the handicapped are appropriately 
and adequately prepared. (Staff in OS BBS and OS BP acknowledge that much work 
remains before CSPD is fully functional.) 

In 1979, under the Education organisation act, a major reorganisation 
occurred for the Bureau of Education for the Handicapped when it became part 
of the Office of the assistant Secretary for Special Education and Rehabili- 
tative Services — its current organisational placement. 

Part D of the Education of the Handicapped Act has remained a cornerstone 
in the preparation of personnel for education of the handicapped for about two 
decades. Likewise, the broad goal of the Personnel Preparation Program has 
remained stable — to train more and letter educators. Beyond that, many changes 
have occurred in program operation throughout the years. These have included 
the training audiences to be served; the content areas of the training; the 
type of training (preservice or inservice); the types of handicapped children 
that personnel are trained to serve; the institutions, organizations, or indi- 
viduals that are eligible to receive training grants; the funding priorities; 
and so on. 



Method of Operation 

The Personnel Preparation Program is administered by the Division of 
Personnel Preparation (DPP) in the Office of Special Education Programs (OSEP) 
within the U.S. Department of Education's Office of Special Education and 
Rehabilitative Services (OSBRS). 

The Personnel Preparation Program provides financial assistance to insti- 
tutions of higher education, state education agencies, and other appropriate 
nonprofit organisations (including parent groups) to conduct activities that 
will increase the supply and improve the quality of personnel who provide 
education and related services to handicapped children. 




-3- 



24 



Financial assistance normally takes the form of grants awarded for up to 
three years, renewable annually. Grantees are Institutions of higher educa- 
tion (XHKs), state education agencies CSSAs), or other appropriate nonprofit 
organisations, and individuals say receive financial support (e.g., student 
stipends) through a grantee. 

The Personnel Preparation Program funds projects thet may include (1) 
training of special education and related personnel to provide instruction and 
other services appropriate for any (or all) types of handicapped children, (2) 
Information and training for parents or persons who work with parents, and (3) 
preparation of degree, nondegree, certified, and noncertlfied personnel. 

The process of focusing program resources on critical needs includes 
these elements: (1) setting priorities, (2) announcing priorities and selec- 
tion criteria annually for funding competition, and (3) reviewing and awarding 
grants. The number of announced priority areas has increased over the years. 
In FY86, OSSRS announced 10 priorities for competition: (1) preparation of 
special educators; (2) preparation of related services personnel; (3) parent 
organization projects; (4) preparation of personnel to provide special educa- 
tion and related services to newborn and infant handicapped children; (5) 
preparation of leadership personnel; (6) special projects; (7) state education 
agency (SEA) projects; (8) preparation of personnel to work in rural areas; 
(9) preparation of personnel for minority handicapped children; and (10) 
regular educators. Hot all published priorities need he announced for new 
grant competition each year; for example, the "transition" priority was not 
announced for new grant competition for FY86. 

Appendix C provides a summary of the funding history for each competition 
area since FY83 for both new and continuation grants. The number to the left 
of each dollar amount is the number of applications funded. 



Rationale f »«- f Federal »ole In Personnel Preparation 

The following discussion provides a context for presenting the objectives 
of the Personnel Preparation Program, and the logic and assumptions underlying 
strategies the program uses to pursue these objectives. 

Why is there a Personnel Preparation Program at allt Wny not leave uni- 
versities, states, and local education agencies (LSAs) » or others entirely to 
their own devices to train personnel to provide education and related services 
to children and youth Who are handicapped? Is there an appropriate role here 
for the federal government? 

Looking at the larger context for the special education enterprise sug- 
gests these broad legal and strategic antecedents: 

• The federal intent, according to P.L. 94-U2. is to ensure a free, 
appropriate public education for all children who are handicapped. 

e It follows that the federal government acts in ways to protect 
handicapped children's right to such education. For example, the 
federal government provides entitlements to states to help offset 
costs of educating all handicapped children (P.L. 93-380). 

e This policy acknowledges that the burden of providing free and 
appropriate education programs for all children who are handi- 
capped is too big for states and LBAs to shoulder without some 
federal assistance. 

e But the federal government's motive is not entirely altruistic. 
Investing federal funds In special education acknowledges a 
national interest in seeing that these children achieve their 
potential for contributing to their own economic well-being, and 
for participating in their community, rather than being strictly 
on social welfare. 

Prom these antecedents, the reasoning proceeds that these unserved and 
underserved children will not have this opportunity unless: 

e There are sufficient n ^h*r« of qualified personnel specially 
trained to provide them the benefits of effective and appropriate 
education. 



a 



The quality of such specialised personnel is sufficient to enable 
children and youth who are handicapped to attain their full 
potential for economic and social self-sufficiency. 
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• The capacity of the system* for personnel asvelopment is suffi- 
cient to meet the above demands for both quantity and quality of 
specially-trained personnel. 

If left to its own devices, the reasoning goes, the system will not 
attain these three aims in a timely fashion nor in a comprehensive enough 
manner. It is assumed that: 

e Special education personnel preparation programs in many univer- 
sities do not attract sufficient numbers of individuals to justify 
costs for program development, improvement, and maintenance. This 
is particularly true for specialities that address unique needs 
of relatively small subgroups of the population of children and 
youth who are handicapped. 

e The same assumption applies to preparation of personnel for 
emerging roles in special education. 

e Without an external stimulus for doing otherwise, model program 
dissemination is likely to be limited geographically, and model 
developers are likely to focus narrowly. Thus, the potential for 
improving personnel preparation practice and, in turn, the quality 
of trained personnel, is limited. 

• Universities will not attract and graduate adequate numbers and 
types of doctoral and postdoctoral leadership personnel to promote 
state-of-the-art practice in personnel preparation at all levels. 

Therefore, external stimuli must be applied to hasten the system in contribut- 
ing to the three aims and to shape the nature and quality of the system's 
response, in short, the appropriate role for the federal government is a 
catalytic one. 

Continuing this line of argument, the federal government is in a uniquely 
advantageous position to stimulate the system to respond to current and emerg- 
ing needs for appropriately trained personnel, model programs, curricula, 
information, etc. For example, the federal government can: 

e Muster resources and information on behalf of the system as a 
whole. 



* This system includes existing and potential training provider-,, resource 
allocators, program developers, R&D institutions, information channels, etc 
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• Provide a national perspective on current and emerging needs (at 
all levels) for particular types of specialized personnel, model 
programs, curricula, etc. 

e Identify and encourage replication of state-of-the-art practices 
in personnel preparation. 

• Maintain national visibility for special education personnel 
development (all levels). 

Accordingly, the Personnel Preparation Program implements strategies that fur- 
ther the aims of increasing the quantity and improving the quality of personnel 
trained to serve children and youth who are handicapped, and of expanding the 
capacity of the system for personnel development. The next section discusses 
each of these strategies and the grant activities through which they are 
pursued . 



Program Goal and Objectives 



The ultiwate goal of the Personnel Preparation Program is: 

• To enhance education and related services for handicapped children 
end youth through the preparation of specialized personnel. 

"Specialized personnel" means an* personnel, including regular educators, 
who have the knowledge and skills necessary to deliver education and related 
services to children and youth who are handicapped. 

Using the word "enhance" in stating this broad goal deliberately implies 

that: 

e Fully achieving "free and appropriate public education" for 

handicapped individuals is beyond the direct control or resources 
of the federal government and, in turn, the Personnel Preparation 
Program. 

e Appropriate roles for the program are complementary and catalytic 
ones, like stimulating new developments and new directions, 
making the "system" work better, and augmenting it, rather than 
substituting for that system. 
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To achieve its ultimate goal within these two caveats and those in the 
authorizing legislation and regulations, the Personnel Preparation Program 
directs its efforts to three "enabling objectives": 

e Produce more qualified personnel to serve children and youth who 
are handicapped. 

e Improve the quality of personnel trained to serve children and 
youth who are handicapped. 

e Expand the capacity of the system for personnel development . 

These objectives are within the direct control of the program. Therefore, they 
provide a useful starting point for examining program strategies, activities, 
and accomplishments in the present goal evaluation. 



Program Logic 

Figure 1 portrays the logic of the overall program. This figure shows 
the relationships among events that influence program design, implementation, 
and capacity to meet the goal and objectives. These events are described 
below. 

Policy Inputs 

Inputs to the program from federal sources include legislation and 
regulations, resources, OSBRS priorities, and a variety of executive agency 
directives. Inputs from "the field" include information and expertise in 
the form of needs analyses and advice from constituencies. 

Program Intuit? 

The above inputs support and help to shape a program of grants to 
eligible institutions and organizations. The grants are for projects in 
priority areas, selected annually for funding in consultation with federal 
officials and representatives of the program's constituencies. 
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POLICY INPUTS 



PROGRAM INPUTS 
(by DPP) 



USING SEVERAL w 
FEDERAL STRATEGIES |^ 



THROUGH GRANT 
PROGRAM ACTIVITIES 



TO ACHIEVE 
PROGRAM OBJECTIVES I 



THAT CONTRIBUTE TO INS 
ULTIMATE PROGRAM GOAL 



i 
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Congress 

• Enabling legisla- 
tion 

P.L. 98-199, Part 0 

a Regulations 
34 CFR, Part 318 

• Appropriations 
$61,250,000 (FY86) 

Executive Aggncieg 

a Administration 
policy directiv s 

a OMB policy direc- 
tives 

a ED policies 

a OSERS priorities 

The Field (Const 1- 
tuencies) 

a Needs analyses, 
data 

a Priority sugges- 
tions 

a Peer review 



a Program support 

a Grant prograa administration 

a CSPD support 

• Leadership and technical 
assistance to the field of 
personnel preparation 
(all levels, as feasible) 

a Coordination and collabora- 
tion with other agencies 
regarding personnel pre-* 
paration 



(T^ Supporting recruit- 
it and retention 



(?) Targeting critical 
needs 

(?) Supporting model 

program development, 
evaluation » and 
dissemination 

(7) Supporting leader- 
ship development 

(b) Encouraging state 
and professional 
standards 

(b) Supporting par ant 
organisation 
projects 

(7) Building capacity 

(s) Promoting institu- 
tionalization 



• Program develop- 
ment, improvement, 
and support, in- 
cluding stipends 

a Model program 
development . 
evaluation, and 
dissemination 

• Technical assis- 
tance and infor- 
mation 



a Produce mora 
qualified 

serve children 
and youth who 
are handicapped 

a Improve the qual- 
ity of personnel 
trained to serve 
children and youth 
who are handi- 
capped 

a Expand the capac- 
ity of the ayatem 
for personnel 
development 



NOTE: Figure 2 shows the relationship among strategies, 
activities, and objectives. 



Enhance education 
and related services 
for handicapped 
children and youth 
through the prepara- 
tion of specialized 
personnel 



Figure l. Personnel Preparation Program Logic Model 
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The Division of Personnel Preparation (DPP) administers the grant 
program, provides leadership and assistance to the field of personnel 
preparation, and (with other units in OSKP), reviews and identifies areas to 
be strengthened in the CoPD component of state plans. Actions taken and 
policies implemented by DPP are supposed to further the program goal and 
objectives. For example, each year DPP develops standards and procedures for 
reviewing new and noncompetin& continuation applications. These guidelines, 
if adhered to, are expected to direct program resources to high-quality 
projects that will produce results which contribute to program objectives. 



Strategies 



The Personnel Preparation Program utilises eight major strategies to 
attain its three objectives. The list below groups the strategies under the 
relevant program objective. The description of each strategy suggests how it 
is supposed to contribute to the program objective. 

e Produce more qualified personnel through: 

1. Supporting recruitment c:d retention ; Funding training grants 
will attract strong candidates who will prepare for, enter, and 
remain in careers in special education, and thereby increase 
the numbers of individuals specially trained to serve handi- 
capped children and youth who are handicapped. 

2. Targeting critical needs : Direct Pogrom resources to 
personnel preparation efforts in areas of critical need will 
vo&ks available more of these types of qualified personnel. 

a Improve the quality of personnel trained through: 

3. Supporting model program development, evaluation, and dissemi- 
nation: Promoting the refinement and distribution of improved 
teaching methods of broad significance, for the field of pro- 
service and inservice personnel preparation (all levels) will 
(a) encourage replication of best practices by other training 
programs, leading to (b) improved quality of personnel trained 
in these programs. 

4. Supporting leadership development : Doctoral and postdoctoral 
preservice training of Individuals who will go on to train 
teachers, do research, and administer programs will (a) 
encourage use of state-of-the-art methods in personnel prepa- 
ration (all levels), leading to (b) improved quality of these 
personnel . 
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efforts of accreditation agencies and professional organise 
tlons to develop appropriately rigorous standards for special- 
iced personnel certlf icati-m, and institutional and/or 
prograsnatic accreditation, and (b) requiring grantees to 
provide assurance that their institutions and proposed 
programs meet such standards, will promote improvements in 
personnel preparation programs that, in turn, will improve 
the quality of personnel trained. 

e Expand syst aa capacity through: 

6. gtnmortln* v*™> organization projects i Providing training 
andinformation to parents will help the m Influence the system 
to develop and exercise its capacity to meet the needs of 
their handicapped children. 

7 Building capacity ; Supporting and encouraging activities that 
increase the system's ability to meet local, state, and 
regional needs for trained and certified personnel, and for 
regular educators qualified to educate handicapped children 
and youth in least restrictive environments, will increase 
system capacity for personnel development (all levels) . 

8. Promoting ing MHifclonalizatlon: Stimulating institutional 
commitments to sustain personnel preparation programs, that 
is, the system's capacity for personnel development at all 
levels, will encourage long-term support for these programs 
after federal support for them ends. 

Grant Activities 

Activities are carried out by grantees, with federal support. Figure 2 
shows the relationship among federal strategies, grant activities, and program 
objectives. These major points are relevant to the two figures: 

e The Personnel Preparation Program pursues the above federal 
strategies through activities that grantees carry out at the 
state, institutional, and local level. (These strategies and 
activities are the row and column labels, respectively, in 
Figure 2.) 

e Thus, the grant programs are the primary mechanism for implement- 
ing federal strategies and legislative intent. 

e It is possible to focus grsnt competitions (for selected prior- 
ities) to accommodate one or more of the strategies. 

• The essential core of grantmaking activity is represented by the 
five clusters of primary activity depicted in Figure 2 (see Roman 
numerals in five cells). 
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PROGRAM OBJECTIVES/ 


Program Development, 
Iaproveaent , and Support , 
Including Stipends 


GRANT ACTIVITIES 

Model Development, 
Evaluation, and 
Dissemination 


Technical Assis- 
tance and 
Informal; 1cm 


Produce more qualified 
personnel . . . 

Supporting recruitment and 
retention 

(5) Targeting critical needs 
areas 


Special Educators CD (2) 
Related Services $ (2> 
Rural (J) w 

Tn f an P U# laV 

Transition 1 (S> © 
Minority I 0 © 






Improve the quality of 
personnel ... 

(3) Supporting model program 
developments evaluation, 
and dissemination 

^4) Supporting leadership 
development 

(?) Encouraging state and pro- 
fessional standards 


I 

Leadership Projects Q> 


Special Projects © 

AMI 




Expand the capacity of 
the system for personnel 
development • • • 

(s) Supporting parent organiza- 
tion projects 

(7 J Supporting Improvements in 
system capacity 

Proaoting institutionalization 


EE 

Regular Educators (2) 
SEA Projects (2> 




Parent Organization 
Projects © 

i 



Figure 2. The Intended Relationship Aaong Prograa Objectives, Federal Strategies, 
Grant Activities, and Primary Foci of Competitions (FY86) 



Cell entries Indicate the main emphasis of FY86* grant activity, by 
priority area. This is a gross classification. The purpose is to show, very 
generally, where the federal investment in grants is supposed to generate the 
most mileage toward one of the three objectives. The matrix conveys the 
expectation that, in amreftate . projects in a particular priority area (1) 
will contribute more to one program objective than to the others, and that (2) 
the means they implement will include the federal strategy (les) "attached" to 
that objective. 

Two of the eight strategies (5 and 8) are not attached to any priority 
area, but this does not imply that nothing is happening in grant projects to 
promote institutionalization and to improve standards . He it her do two empty 
cells in the row for recruit and retain (Strategy 1) and for targeting 
critical needs (Strategy 2) imply that the program is unlikely to attain its 
objective of increasing the numbers of qualified personnel available to serve 
children and youth who are handicapped. 



* These clusters might be constituted differently, depending upon how each 
competition area is defined for a particular fiscal year. For example, in 
the first year of funding for the Rural priority, the competition focus was 
model development (Strategy 3). Therefore, it would not be unusual today to 
find a continuation project in that priority area that emphasizes this 
strategy rather than teacher training (more relevant to Strategies 1 and 2). 



Causal Assumptions 



Explanatory statements in the above list of eight strategies strongly 
imply the cause-effect linkages between each strategy and one of the program 
objectives, and are not reiterated here. 



Another set of assumptions relates to the grant activities through which 
these strategies are pursued. These activities (which are the column labels 
in Figure 2) and their related assumptions are as follows: 



• Program development, improvement, and support, including stipends , 
will stimulate the system to produce more qualified personnel to 
meet current and emerging needs of handicapped children and 
youth* and will make such personnel available in a more timely 
fashion. 

Providing stipends to strong candidates for careers in special 
education will help dissuade them from investing in other career 
preparation options and will increase the likelihood that they 
will enter and remain in special education to provide services to 
handicapped children and youth, to train others, and to lead 
efforts to expand and improve the system for personnel develop- 
ment (all levels). 

• Model development, evaluation, and dissemination of best prac- 
tices will stimulate the field of personnel preparation to 
implement such exemplary approaches, which in turn will make 
available more high-quality personnel to deliver services to 
handicapped children and youth. 

a Providing technical assistance and information to training pro- 
viders, including parent organisations, stimulates improvements in 
training and system capacity that make available more personnel 
and parents who are able to provide effective education and 
related services to handicapped children and to interact effec- 
tively with the system on their behalf. 



II. METHODOLOGY 



Data collection for the goal evaluation of the Personnel Preparation 
Program took place during August, September, and October, 1986. its purpose 
was to obtain information about the inputs, strategies, and grant activities 
that are being carried out and supported to achieve the federal program 
objectives that were described in the previous chapter. 

The study team conducted detailed project reviews for a sample of projects 
selected to represent the essential core of the Personnel Preparation Program's 
grant activity. "Essential core" is defined as the five clusters of primary 
activity depicted in Figure 2 (Section I) . 

Data collection included reviews of a representative sample of 57 projects, 
selected as described below. Each member of the study team was responsible for 
a specified number of the projects selected, and for following a protocol 
(Appendix A) to complete project reviews. Bach review assembled information on: 

e the basic parameters of the project (e.g., focus, competition 
area, agency type, funding history, staffing); 

e the nature of grantee activity and target groups, including 
institutional and state contexts, as appropriate; 

e implementation of federal strategies through grantee activity; 

e the intended logic of the project (e.g., the proposed linkages by 
which project activities will lead to the attainment of desired 
results, and the linkages by which grantee activity is intended 
to further the objectives and ultimate goal of the Personnel 
Preparation Program); 

e any changes that have taken place in project plans since the 
latest grant award; 

e evidence of project performance to date (e.g., personnel trained, 
models produced and disseminated, technical assistance provided); 

e evidence of project institutionalization or system capacity 
building (e.g., extent to which federally .unded activities will 
be picked up by nonfederal sources at the end of the project); 

e permanent organizational changes that have occurred as a result 
of the project}; 
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major constraints experienced, addressed, and anticipated; 

the process by which the grant was negotiated and awarded and 
through Which project performance has been monitored since award; 

implementation by DPP of other processes ("program inputs") that 
are related to the project and its competition area; and 

grantee and DPP staff perceptions of the extent and quality of 
federal "program inputs" that are relevant to the project and its 
competition area. 

obtain the above information, the reviewer consulted several sources 

the initial program solicitation leading to the grant award 
(e.g., FY 1984 grants announcement); 

initial and continuation applications (e.g., FYs 1984, 1985, 
1986, as available); 

technical review/ evaluation and award documentation; 
monitoring reports if available; 

documentation of results of grant projects (e.g., data on the 
previous year's accomplishments which are appended to the 
beginning of a continuation application, or which may be 
described in it); 

► 

products or deliverables from the grant project; 

telephone interview with the grant project director or principal 
investigator (75 minutes was the average length of an interview); 

telephone interview with the DPP competition manager; end 

literature and other selected sources that were relevant to the 
project or its competition area, to its Institutional or state 
context, or to presenting findings of the goal evaluation for 
clusters of federal strategies and grantee activities. (Examples 
included the latest 1986 University of Maryland survey of special 
education personnel supply and demand, materials provided by the 
project officer for the Rand study of teacher supply-demand, 
Center for Statistics data summaries, and materials prepared by 
professional organisations or previous Personnel Preparation 
Program grantees that were relevant to CSPD activities in states 
and/or to improving the quality of personnel preparation programs. 
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As is apparent, project reviews ware limited to secondary sources of 
information. Primary data collection was beyond the scope and resources of 
the goal evaluation (but will be possible during the strategy evaluation phase 
of the study). 

A separate point is that, of all of the above sources, telephone interviews 
with project directors or principal investigators provided the most up-to-date 
information on project activities and accomplishments, and on the nature of 
supporting data that final performance reports were likely to contain. The 
study team did not go on site to examine project records, nor did the goal 
evaluation schedule and budget make it feasible to obtain independent third- 
party verification of information project staff conveyed in the interviews. 
However, the study team did check information the interviews provided against 
other secondary data (e.g.. initial and continuation proposals, phone monitoring 
reports). There were no serious inconsistencies. 



Sample Selection 

A stratified sampling approach was used to be sure that each competition 
area was represented in the projects that could be reviewed. These strata 
corresponded to the competition areas ("priorities") for grants from the 
Personnel Preparation Program. 

The numbers of projects to be sampled from each competition area were 
determined in consultation with DPP staff, according to the ease or difficulty 
of capturing the variability of projects considered to be "true" specimens 
within that competition. The sampling pool for each subsample was determined 
according to procedures described below. The pools and subsamples consisted of 
the following numbers of cases for each stratum: 
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12 


25.0 




18 


27.7 




8 


37.5 


7 


40 


17.5 


7 


21 


33.3 


4 


** 




4 


** 




5 


22 


22.7 



Special Educators 
Related Services 
Rural 
Infant 
Transition 
Minority 

Leadership 
Special Projects 

Regular Educators 
SEA Projects 

Parent Organization Projects 
(including the TAPP prime and 
one TAPP regional subcontract) 

TOTAL CASES IE SAMPLE 57 

In general, the sample was drawn randomly from each stratum according to 
these three steps: 

1 For each of the competition areas, continuations whose initial 
year of funding was FY 1984 were identified. This constituted 
the sampling pool. 

2 Using a table of random numbers, subsamples were drawn from the 
pool in the quantities above, and additional random selections 
were drawn from which to replace cases, if thlsM>acame necessary 
to achieve representativeness. 

3. DPP staff reviewed selections, deleted anomalous ones, and 
replaced them in sequence from the randomized lists compiled 
in Step 2. Reasons for eliminating particular cases and for 
substituting others are summarized in Appendix 0. 



* The 27 projects in the Related Services pool represented these specialists, 
or specialty areas: j-araprofessional (M-9); therapeutic recreation (H-8); 
occupational therapist, physical therapist, nurse («-*); career, employment 
habilitation (M-3); and school psychologist (M-3). 

** See explanation for "R««ular Bdu f afc««^/SRA Projects" on page 19. 
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Stops 1 and/or 2 were modified as follows to select projects from eight 
of the competition areas (priorities): 



e Related Services (H«5). One project was randomly selected from 
each of five major occupational specialty areas* that were 
identified from scanning titles of FY8* continuations that were 
initially funded in FYS*. 

e Regular Bducators/SSA Projects (lfr»4 each) . The plan was to 
identify states that had both types of grants, and then to draw 
four of those states at random. However, only three states met 
this criterion. Because these three states ranged from small to 
moderate in size, the fourth project of each type was drawn from 
the largest state possible in each case. 

e Parent Organization Projects (H«5) . This subsample consisted of 
the TAPP prime contractor, one of the regional subcontractors, 
and three parent projects. The subcontractor was chosen at 
random, as were the three parent projects. 

e Rural. Infant. Transition. Minority (H-16 in all) . Only the 
"Transition** competition included continuations whose initial 
year of funding was FY 1984. However, about three dozen FY86 
continuations under "Special Educators" appeared to focus on one 
of these four current priority areas, and were initially funded 
in FY 1984. (This was determined by reviewing with the DPP 
Director titles and GOO numbers of continuation proposals for 
Special Educators grants that are listed in the FY 1986 Grant 
Award Characteristics Report printout.) These projects constitu- 
ted the sampling pool, augmented by adding continuations under 
the Transition priority whose initial year of funding was FY 1984, 
and continuations under the Rural priority whose initial year of 
funding was FY85. Then Rural, Infant, Transition, and Minority 
projects were drawn randomly from their respective subsets. 



* See footnote on previous page. 
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Rationale f or the Sampling Approach 



Given the limited resources for data collection in a goal evaluation, it 
is not possible to achieve statistical power however the sample might be con- 
structed, for a program the size of the Personnel Preparation Program. In 
these circumstances, sampling is not Intended to get some true population 
value, but is designed to yield ideas, insights, and understandings that will 
permit inferences about how federal strategies are implemented through various 
kinds of grantee activity, under what conditions, with what results, and with 
what implications for program plausibility and performance. 

Therefore, the approach to selecting projects ensured that the sample: 

e covered the various sets of projects (and agencies) that engage 
in a particular type of grantee activity and that represent one 
or more of the federal strategies of the Personnel Preparation 
Program; 

a included projects that fit well in a given cluster or competition 
area; 

a represented different types of grantee experience; and 
a did not include anomalous selections. 

Limiting sample selection to continuations funded originally in FY 1984 
ensured that project reviews would have an opportunity to look at functioning 
operations for which there was a reasonable chance that outcomes, and reports 
of those outcomes, would have been produced. Restricting the sample to proj- 
ects that were currently operating, and that began in FY 1984, also ensured 
that they would have been running long enough to have learned lessons from 
their implementation experience that would be very informative for the goal 
evaluation. Finally, better cooperation was expected from projects that were 
currently operating than from projects that had been completed or discontinued. 

The rationale for selecting more projects from some competition areas 
than from others (e.g., 9 from Special Educators, 3 from Infants) was that it 
would have been harder to capture the variability of projects in some competi- 
tions than in other competitions. 
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Drawing continuation projects readonly from competition areas as a first 
pass at sample selection assumed that there was no reason to expect that cer- 
tain types of projects would be seriously overrepresented or underrep resented 
in a particular competition area. The exception was Related Services person- 
nel, for which the sampling procedure was modified (see above). 



Providing for review by DPP staff of the randomly drawn subsamples to 
check for representativeness and to make purposive adjustments recognised that 

e there is variability within competition areas, and that a program 
manager is concerned with information at several levels (e.g., at 
the cluster(s) level, at the competition/priority level, and 
within the competition); 

e since institutions of higher education (XH8s) may get multiple 
awards from the Personnel Preparation Program, the random draw 
could select several projects that are in a single IHB (and, in 
turn, a single department of special education); 

e much of the variation in projects may be related to the size of 
an IHB's special education program, and the random draw may not 
achieve a desirable balance between large and small IHBs (and, in 
turn, special education departments); and 

e to the extent possible, states represented in the sample should 
be geographically distributed to cover major regions of the 
country . 



Data Collection and Analysis 



Once an acceptable set of 57 projects was selected, the study team mailed 
letters to the grant project directors or principal investigators explaining 
why their cooperation was being sought, and began project reviews according to 
the protocol and instrument in Appendices A and B, respectively. File reviews 
and interviews were conducted on a confidential basis, and grantees were 
assured that the goal evaluation report would not identify specific projects 
for which findings were applicable. 
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The data base consisted of 56 completed project review instruments,* 
compiled by members of the study team according to the protocol, each coded 
with an identification number to facilitate assembling data within and across 
subsamples. To aggregate and analyse this very large compilation of 
information— some in narrative notes, some induced to checklists— the study 
team followed the steps below. 

Step 1. Identify prominent aspects of grantees* 
implementation ol m eight federal strategies. 

Bach study team member reread the project review Instruments they tad 
completed for their particular set of assigned projects, and developed cate- 
gories for individual projects that would capture prominent aspects of bow 
that grantee had actually implemented one or more of the strattgieb. Although 
the study team did not have time to read each other's notes, or to cenduct 
lnterrater reliability checks, they frequently discussed the categories they 
were developing, and agreed on wording that would facilitate eventual aggrega- 
tion within and across subsamples. 

The study team also worked out how to judge when a project did or Hid not 
fit a category, and if a strategy was or was not being "emphasized.** This 
negotiation process was ongoing and represented a significant investment of 
thought and time. The rough ground rule was this. Strong elements of the 
federal strategy had to be evident from both of the following: (a) descrip- 
tions of specific efforts or activities that indicated how the strategy was 
being implemented (provided by the interviewee and project documents); and (b) 
supporting data or Information that the project was collecting and was likely 
to include in its final performance report. Grantees, for example, frequently 
perceived that they were emphasizing model development, evaluation, and 
dissemination (Strategy 3), when in fact strong elements of this strategy were 
lacking (very little effort made with regard to model evaluation, 
dissemination, or both). 



* one of the 57 projects was dropped because available information was too 
minimal to include it in subsequent analyses. This project was among seven 
projects selected from the Leadership competition area. 
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SfcSP 2, Identify or<H«ct results and the 
nature of grantees' mioportlna evidence. 



The procedure followed in Step 1 was also applied in dolus Step 2. Notes 
in the project review instruments that described project accomplishments and 
data sources were reexamined to develop categories to describe (1) the 
specific nature of these accomplishments and (2) the type of supporting data 
that grantees were collecting and were likely to report at their project's 
conclusion. Again, study team members Interacted frequently to refine their 
categories and to agree on conventions for judging whether a project fit a 
category . 

Step 3. Prepare "Preliminary Data summaries. " 

When the study team had completed Steps 1 and 2 for six of the eleven 
competition areas, they assembled the information for presentation to the work 
group. The purpose was to give them a preview of the quality and quantity of 
information in the data base for subsequent use in the plausibility analysis 
and in estimates of prospects for attaining program objectives. 

Step 4. Summarize findings at all levels of 
Interest to Pe rsonnel Preparation Program managers . 

Working from the Preliminary Data Summaries (Step 3) for the partial 
sample, the task leader made a first cut at summarizing findings at three 
levels: for each competition area; for the predefined clusters of competition 
areas (the five filled cells in Figure 2); and across all (56) projects in the 
study sample. The summaries were in chart form, with columns left blank for 
the five competition areas that had not been Included in the Preliminary Data 
Summaries . 

After refining these draft charts in consultation with the study team, 
the task leader and the rest of the team filled in remaining data for their 
respective projects in the five remaining competition areas. 

The task leader decided to lay out the findings this way to provide a 
picture that would be useful to federal program managers. DPP's director, 
branch chiefs, and competition managers are not only interested in findings 
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for the program as a whole, hut also for units and suhunits of the program 
(cells and competition areas). 

Step 5. Ub« tK> data a j *h« baals for aaseeslng 
^l^tatl o n of federal strategies, proapective 
program performance, and program plausibility;. 

a. Section III describes procedures used to analyze^lementation 
of the eight federal strategies by projects in the study sample. 

b. Section IV explains procedures used to make preliminary 
estimates of prospective program performance in attaining 
Personnel Preparation Program objectives. 

c. Section V defines and describes the plausibility analysis. 



Caveats 



The remaining chapters in this report present the goal evaluation 
findings, draw conclusions from them, and propose areas that might be 
considered for further study during the strategy evaluation^?. 

The study team tried to be judicious in its conclusions, bearing in mind 
the restrictions inherent in the methodology. These methodological reminders 
are reviewed now for the benefit of the reader, and the same points are 
repeated 1-ter in this report, where appropriate: 

a The decision to draw the sample from "live" projects necessarily 
restricts the study to conclusions on prospective program 
performance, supported by data that projects are now collecting 
and are likely to present in their final performance reports. 

• it is not within the scope of a goal evaluation to collect 
primary data on project accomplishments, or to capture all 
relevant perspectives. Project reviews rely on two *»i° r 
secondary data sources: initial and continuation applications in 
grant files, and interviews with project directors or principal 
investigators.* although interviews were conducted on a confi- 
dential basis, and most interviewees seamed to be candid, it is 
possible that some relevant information was not communicated. 



* Third-party evaluations of grant projects were very rare. Where such secon- 
darydata were available, they were also included in the project review. 
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• The goal evaluation sample is email in proportion to the sice of 
the program* although it le representative of the broad array of 
Personnel Preparation Program grant activities, ana 6 of the 11 
suhsemples constituted between 25% and 37% of their sampling 
pools. 

e Evaluation resources for this study did not permit data collec- 
tion from third parties, such as consumers (agencies who utilize 
personnel trained and models or programs developed through grant 
activities). They could have indicated the extent to which these 
products are meeting their critical needs and are found to be 
high-quality, useful, and effective. 

e The goal evaluation examines federal strategies as the Personnel 
Preparation Program perceived them, and grantees implemented 
them, in grant activities operating in FY86. 

e This goal evaluation did not examine program management procedures 
per se, but it did try to determine whether intended major program 
inputs (see Figure 1 in Section I) occur at a level that supports 
program objectives and the federal strategies that are pursued to 
attain objectives. 
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III. PROGRAM IKPLKHBMTATIOM 



Introduction 

One major purpose of the goal evaluation for the Personnel Preparation 
Program was to determine the extent to which federal strategies are actually 
being implemented through grant activities. 

Figure 2 (Section I) indicated which strategies the federal program 
expects to be the primary emphasis of particular competition areas. These 
federal expectations allow for the fact that projects may emphasise strategies 
in addition to the primary one(s). 

The following analysis of program implementation is pertinent to three 
columns of the logic model in Figure 1 (Section I): 

• federal program inputs by the Division of Personnel Preparation 
(DPP) (Column 2); and 

• strategies pursued through grant activities (Columns 3 and 4). 

The order of presentation treats strategies first and federal inputs second. 
Each presentation addresses these questions: 

What are grantees and DPP attempting with respect to these inputs, 
strategies, and activities? 

What evidence do they claim to have that confirms that these 
processes are operating? 

What are constraints on implementation? 

is there reasonably good alignment between intended and reported 
program operation? 

Answers to these questions provide part of the basis for estimating the 
likelihood that the Personnel Preparation Program can achieve its objectives. 
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Before proceeding, the reader should have in mind the procedures the 
study tees followed to decide whether a project was or was not "emphasizing" a 
particular federal strategy. These procedures (explained in the previous 
chapter) were as follows: 



The rough ground rule was this. Strong elements of the federal 
strategy had to be evident from both of the following; (a) descrip- 
J tions of specific efforts or activities that indicated how the 

strategy was being implemented (provided by the interviewee and 
project documents); sod (b) supporting data or information that the 
project was collecting and was likely to include in its final 
performance report. Grantees, for example, frequently perceived 
that they were emphasizing model development, evaluation, and 
dissemination (Strategy 3), when in fact strong elements of this 
strategy were lacking (very little effort made with regard to model 
evaluation, dissemination, or both). 



Frequency counts for findings in thj balance of the present chapter 
duplicative; that is, a project could Implement a particular strategy in 
then one of the ways indicated in the text or tables. 



Strategy 1: R ecruitment/Retention 
Review of Federal Expectations and Assumptions 

In supporting recruitment and retention, the Personnel Preparation Pro- 
gram expects that grantees will attract strong candidates who will prepare 
for, enter, and stay in careers in special education, thereby Increasing the 
numbers of individuals specially trained to serve handicapped children and 
youth. The relevant federal objective is to "produce more qualified personnel 

The major grant activities through which the Personnel Preparation Pro- 
gram pursues this strategy are "program development, improvement, and support 
(including stipends)." By providing these funds, the federal program expects 
to stimulate the system to produce more qualified personnel in a more timely 
fashion than would be the case without this extra incentive. 
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According to the relationships presented in Figure 2 (Section I), the 
Personnel Preparation Program expects that rocrultaont and retention will 1 
major emphasis of grants funded under competition areas in Cell _\. 



Findings 

All projects in the study sample for Cell I were judged to be emphasising 
recruitment and retention . (The abbreviations below appear in the summaries 
of findings throughout this chapter, lis are sample sizes, used in reporting 
frequencies for findings.) 

Special Educators (SPED, H»9) 
Related Services (RBL, M=5) 
Rural (RUR, M-5) 
Infant (IMF, »»3) 
Transition (TRANS, H=5) 
Minority (MIS. H«3> 

Overall, these projects tried to recruit well -qualified candidates who 
demonstrated a strong Interest in the area of need addressed by the grant. 
They offered stipends to attract strong candidates and relied on practicum 
experiences to promote retention and enhance commitment to special education 
roles. In some cases, candidates were already working in such roles and, 
therefore, committed. Few, if any, retention activities were required for 
such individuals. 

In addition, all projects in Cell II (the Leadership competition area) 
emphasized recruitment . They recruited extensively at other universities, 
both in and out of state, used a variety of methods to promote their programs, 
and deemed stipends to be essential in attracting full-time, high-quality 
doc tor *1 and postdoctoral candidates. 

Projects in the Leadership subsample did not perceive a need to emphasise 
retention strategies, as such. They reasoned that if people have made it as 
far as a doctoral program, they are committed to careers in special education. 
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Table 1 

How Grantees Implemented Recruitment/Retention (Strategy 1) 



gumbor of Pro 4 acts (Cell I) 

Description SPED RKL RUB INF TRA HIV 

H-9 S-5 feg Bs3 gn§ B»3 

Recruited extensively, often at other 3 14 3 2 
universities, both in and out of state 

Offered stipends to attract qualified trainees 8 4 3 3 4 1 

Emphasized attracting strong candidates 4 5 3 

Recruited trainees from rural areas 1 5 

Promoted programs, increased awareness of 4 1 2 4 

needs area at conferences and workshops 

Assessed seriousness of candidates' interest 3 3 

in the needs area 

Emphasised practicum experiences to promote 4 4 3 3 

retention and enhance commitment to needs area 

Routinely provided updated information on 2 
specific job openings in needs area 

"Retention" activities not necessary; many 11 3 

students already in service roles and thereby 

committed 

Active recruitment of minorities 1 2 

"Recruitment/ retention" activities minimal; 1 
jobs readily available in state for graduates 
of program 

Program staff serve as advocates for students 1 l 
to help them obtain jobs 

High reputation of program and university 1 
attracts students; recruitment minimal 
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Grantees indicated that they have the following supporting data for this 
federal strategy: 

numbers of candidates recruited 

Candidates* test scores and grade point averages 

Teacher/supervisor/ administrator recommendations 

Selection criteria and protocols 

Reports from practlcum supervisors 

Bumbers of trainees who remain in special education after 
the program 

Constraints 

Major constraints on recruitment and re tention were financial. 

For orolects in Cell I (SPED. RBL. BU B. MIH. TRA. IHF): 

Stipend allowances were not necessarily a sufficient incentive for 
qualified candidates with financially attractive options. 

This was a particular problem in recruiting qualified minority 
trainees . 

Given insufficient stipends, trainees were likely to work during the 
day, take courses at night, and have difficulty keeping their grades 
up to expected levels. 

Because of critical shortages, uncertified teachers are hired, and 
this reduces che incentive for potential trainees to enter 
certification programs. 

"Minority" projects found that it was difficult to recruit minority 
trainees to geographical areas or universities where there were few 
minorities. 

For projects in Cell II (LPR) : 

High tuition constrained how many candidates doctoral and post- 
doctoral programs could recruit. 

In being very selective, doctoral and postdoctoral programs 
sometimes did not get as many qualified people as desired; if not, 
they intensified recruitment. 

It was especially difficult to recruit strong minority candidates 
because these individuals have a lot of options. 
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Strategy 2: Targeting Critical Reads Areas 



Review of Federal expectations anU Assumptions 

The Personnel Preparation Program expects that directing program resources 
to training in areas of critical need will stimulate the field to prepare more 
of these types of personnel. The relevant federal objective is to "produce 
more qualified personnel." 

The major grant activities through which the federal program pursues 
this strategy are "program development , improvement, and support (including 
stipends)." By providing these funds, the Personnel Preparation Program 
expects to stimulate the system to produce more qualified personnel to meet 
current and emerging needs of handicapped children and youth, and to meet 
these needs in a more timely fashion than would otherwise be the case. 

According to the relationships presented in Figure 2 in Section I, the 
federal program expects that targeting critical needs areas will be a major 
emphasis of grants funded under competition areas in Cell I . This relation- 
ship, therefore, is identical to that assumed for Strategy 1 (Recruitment and 
Retention) . 

Findings 

All projects in the study sample for Cell I were judged to be emphasizing 
targeting critical needs areas . Overall, these projects used a variety of 
data on needs to make a strong case for the grant's focus on a particular role, 
choice of training approach, and content of training. Even if the quality of a 
single data source improves (e.g., CSPD, annual state counts), grantees believe 
that it will continue to be necessary to use a variety of data sources in order 
to focus their activities with sufficient precision, and to make a sufficiently 
persuasive case for proposed activities. 

Similarly, all other projects (Cells II through V) were targeting critical 
needs areas, and implementing this strategy in the same manner described above. 
That is, they selected and used data in ways that justified the major emphasis 
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of their proposed activities, whether these were to be program development and 
improvement , model development and dissemination, or training and technical 
assistance. 

Table 2 shows the various sources that grantees used to document the 
critical needs that their projects proposed to address. In general, needs 
statements and supporting documentation were convincing, and reviewers in all 
competition areas frequently said they were among a proposal's strong points. 

Evidence that suggests the extent to which grantees are implementing this 
federal strategy includes: 

Excerpts in grant applications from a variety of data sources , including 
CSPD and other state data 

Fit between proposed grant activities and documented critical needs areas 
Constraints 

The major constraint on targeting critical needs areas was the inadequacy 
or unreliability of any single data source for justifying the grantee 9 s 
proposed project, focus, and approach. 

CSPD information and states* annual counts were least useful for docu- 
menting needs for highly specific personnel specialties (e.g., in Related 
Services, Transition, and Infant competition areas), and for projects serving 
regional and national interests or needs, such as those submitted under the 
Leadership amd Special Projects competitions. 

The quality, accuracy, and timeliness of information in states' CSPOs 
varied widely. 

There were no central data bases relevant to preparing personnel to serve 
handicapped individuals in out-of -school settings (e.g., hospitals). 

Equipment costs (e.g., several thousand dollars for "Versabraille") were 
a financial constraint for projects preparing personnel to meet the needs of 
some populations, such as blind, deaf, or multiply handicapped individuals. 
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Table 2 



Data Sources Grantees Used to Document Critical Needs 
That Their Projects Proposed to Address 



Pat a Source/Description 






Cell 


I 






Cell II 


Cell HI 


Cell 


IV 


Cell V 




Tot* 


ils by Cell 




Overall 


SPED 


REL 


HUH 


INF 


TRA 


MIN 


LDR 


SPROJ 


REC 


SEA 


POP 


X 


11 


III 


IV 


V 


(N-56) 




N-9 


N-3 


N-5 


N*3 


N«5 


N-3 


N-6 


N-7 


N-4 


N~4 


N-5 


N-30 


N-6 


N-7 


N-8 


N-5 


N 


CSPD data and priorities 


4 


1 


1 


1 


4 


2 




7 


3 


2 


2 


13 




7 


5 


2 


27 


Extrapolation from CSPD 
data* 














4 












4 








4 


Other state data and 
priorities 


9 


3 


2 


1 


4 


2 


5 


7 


4 


4 




21 




7 


8 




41 


Regional /nat*<jnal data 
and priorities 


2 


3 


2 


3 


4 


I 


5 


b 


2 




1 


15 


5 


5 


2 


I 


28 


Local data and priorites 


4 


1 


1 


I 


1 


2 


4 


2 








10 


4 


2 






16 


Literature 


8 


5 


1 


1 


4 




3 


b 


3 


4 


2 


19 


3 


6 


7 


2 


37 


Experts 8 consensus 


7 






1 




1 


3 


2 






2 


9 


3 


2 




2 


16 


Professional organizations' 
survey data 






i 








1 


1 








1 


■ 

1 


1 






3 


University's (IHE's) survey 
data or other information 






I 




I 




4 










2 


4 








6 


Accreditation organizations 














1 












1 








1 


Status of existing training 
oppor t unities 






1 






1 




5 




1 




2 




b 


1 




8 


Potential consumers*, e,g, , 
potential employers of 
personnel trained; poten- 
tial users of training 
models, materials, 
produced 


7 


I 


1 








2 






2 




9 


? 




2 




13 


Survey of eir rs of 
entry- leve: ^le 
Without handicaps 




















1 










1 




1 



* CSPD does not address needs for leadership personnel, only for direct service personnel* Needs for leadership personnel must be extrapolated 
from other data on teacher shortages/students served* 



Tine was another constraint. Interviewees frequently observed that it 
takes years to develop and refine personnel preparation programs to meet 
critical needs areas, and that grant staffs invest what one project director 
called considerable "out-of-hide" time in program development and improvement. 

Uncompetitive salaries and unappealing geographical location were two 
problems (not amenable to training program interventions) in targeting 
critical needs areas. 

Strategy 3: Model Development. Evaluation, and Dissemination 
Review of Federal Expectations and Assumptions 

In supporting model development « evaluation, and dissemination , the 
Personnel Preparation Program tries to promote refinement and distribution of 
improved personnel preparation methods that have broad significance for the 
field of inservlce and preservice preparation. The assumption is that devel- 
oping and disseminating exemplary practices will stimulate the field to 
implement the approaches, which in turn will make available more high-quality 
personnel to deliver services to handicapped children and youth. The relevant 
federal objective is to "improve the quality of personnel." 

According to the relationships presented in Figure 2 in Section I, the 
federal program expects that model development, evaluation, and dissemi nation 
will be a major emphasis of grant activities in Cell III , the competition area 
known as Special Projects. 

Findings 



All projects in Cell III (the Special Projects competition area) were 
judged to be emphasizing model development, evaluation, and di ssemination. 
Grantees reported that they emphasized this strategy in these ways: 



Model development: 

Incorporated new or innovative elements* in their training models 
(7/7) 9 which most frequently included; 

cross-department , cross-agency 9 multidisciplin&ry, or total 
building staffing (5/7) 

new information, new experiential activities (5/7) 

videotape portrayals to convey informational, experiential t or 
emotional content (3/7) 

Incorporated state-of-the-art practice* knowledge, and/or proven 
approaches in their training models (7/7), and typically identified 
such features by one or more of the following means: 

lessons learned from earlier (pilot) efforts (4/7) 

consultation by nationally recognized advisors, specialists, 
experts (4/7) 

research-based literature on staff development in the appropriate 
needs area (3/7) 

Developed * raining models as components of more comprehensive 
intervention models (5/7) 



Model evaluation : 

Obtained qualitative, subjective feedback from trainees, their 
supervisors, and project staff (5/7) 

Emphasised formative evaluation in the context of the model's 
developmental try out and revision (4/7) 

Determined actual use of model practices after training (4/7) 

Assessed trainee performance an pre-established, specific, behavioral 
competencies and training objectives (3/7) 



Model dissemination : 

Emphasised promoting awareness of the model training program (7/7) 

Targeted a broad array of potential users and service delivery 
settings (7/7) 

Emphasized staff development workshops for potential consumers of the 
model training program (4/7) 
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Conducted model demonstration/dissemination as part of the model's 
developmental try out (4/7) 



Targeted potential users In other states (3/7) 

In addition, some projects in other competition areas were judged to be 
emphasizing model developmer t . evaluation, and dissemination , according to the 
pattern noted above for Cell III: 

Special Educators (2/9) 
Sural (2/5) 

Regular Educators (2/4) 
SEA Projects (3/4) 

Many more projects than the above would have counted themselves as emphasizing 
this federal strategy. However, the study team's criterion was that a project 
must prominently exhibit strong elements of the three aspects of the strategy. 
Many grants whose major activity was program development, improvement, or 
support did not meet that criterion, nor were they expected to. 

Grantees indicated that they have the following supporting data for this 
federal strategy: 

Model descriptions and materials 
Consultant vitae 

Description of experts' roles in development 

Evaluation instruments and summaries 

Lists of training objectives and competencies 

Number, type, and reach of dissemination activities 

Mumber and nature of implementations of models by others 

Number of written requests for information, training 

Constraints 

Grantees encountered financial constraints in model development , such as 
very high costs for professional production of videotapes, and for computer 
equipment for trainees. 
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Model evaluation was hampered by the lack of agreed-upon professional stan- 
dards for personnel who serve some target groups, e.g., infants and newborns, 
and the absence of valid and reliable quantitative measures of competence. 



Model dissemination consisted of promoting awareness and demonstrating 
the model approach at workshops. Assistance to consumers in replicating 
models was not possible within the time and resources available for these 
three-year grants. 

The study team also observed that, with very few exceptions, it was 
unlikely that grantees would be able with grant funds to package their models 
in a format convenient for widespread dissemination. 

Strategy 4; Leadership Development 

Review Federal Expectations and Assumptions 

By supporting doctoral and postdoctoral preservice training of individ- 
uals who will go on to train teachers, do research, and administer programs, 
the Personnel Preparation Program expects to encourage use of state-of-the-art 
methods in personnel preparation (all levels) which, in turn, should improve 
the quality of these personnel. Therefore, grant activities funded under the 
Leadership competition area are expected to contribute most to the second 
program objective. 

The major grant activities through which the Personnel Preparation 
Program pursues this strategy are "program development, improvement, and 
support (Including stipends).** By providing these funds, the federal program 
intends to stimulate the system to produce more high-quality personnel in a 
more timely fashion than would be the case without this extra Incentive. 

According to the relationships presented in Figure 2 in Section I, the 
Personnel Preparation Program expects that leadership development will be a 
major emphasis of grant activities in Cell II . the competition area known as 
Leadership. 
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Findings 



All projects in the study sample for Cell II emphasized leadership devel- 
opment . Grantees reported that they Implemented this strategy as follows: 

Clearly specified the roles and functions to be performed by 
program graduates (6/6) 

Incorporated state-of-the-art practices in leadership development 
(6/6) 

Included coursework, practice, internship, and dissertation 
experiences in leadership training (6/6) 

Program faculty served as mentors and models (6/6) 

Students worked with faculty in a research "apprenticeship" (3/6 > 

Project-level data that support implementation of this federal strategy 
include: 

Course and program descriptions 

Student records of course completion, performance 

Constraints 

Grantees in the study sample did not indicate major constraints in 
implementing leadership development . 

strategy 5: State and Professional Standards 
Review of Federal Expectations and Assumptions 

This strategy was not expected to be a major emphasis of grantee activity. 
However, all applications to the Personnel Preparation Program for preservice 
training grants must provide assurance that "the proposed project meets State 
and professionally recognised standards." In this way, the federal program 
seeks to encourage improved training which will, in turn, improve the quality 
of personnel prepared through these programs. 
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Findings 



A few projects in each of the following competition areas have undertaken 
activities that go beyond the "letter" of statutory requirements and suggest a 
forceful emphasis on state and Professional standards as a strategy for improv- 
ing the quality of personnel trained: 

Special Educators (SPED, 3 of 9) 
Related Services (RBL, 3 of 5) 
Transition (TKA, 1 of 5) 
Leadership (LOR, 3 of 6) 
Special Projects CSPROJ, 1 of 7) 
Regular Educators (REG, 2 of 4) 
SEA Projects (SEA, 2 of 4) 

In all, these IS projects represented seven competition areas in Cells I, 
II, III, and IV of Figure 2 (Section I). 

Table 3 presents information on activities these projects have undertaken 
in a deliberate effort to promote state and professional standards . Project- 
level data that they claim will substantiate this emphasis include: 

Assessment instruments that reflect standards 

Monitoring forms that Incorporate standards 

Training content that is visibly and substantively 
consistent with standards 

Grantees did not Indicate major constraints in implementing this federal 
strategy, but in a couple instances commented that standards applicable to new 
or emerging fields or to new priorities have yet to be fully developed or 
widely accepted. 
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Table 3 



Avtlvltles T hat Grantees Said They Undertook to Promote 
state/Professional Stand ards (Strategy 5) 



HUmber of Projects 



Description SPED RBL TRA LDR SPROJ REG SKA 

g»9 B»5 H=5 g-6 H°7 g-4 »*4 

Used professional standards as guidelines 13 13 1 
for training or for model development 

Used state standards as guidelines for 2 11111 

model development, or for preparing 
personnel for certification 

Faculty members or project staff served 2 2 11 
on boards of standards-setting agencies 
or organizations 

Competencies specified in program model 1 1 

will be incorporated in the monitoring 

form SUA uses to review university-level 

special education personnel preparation 

programs 

Program competencies served as model for 1 
professional guidelines for training 



NOTE: All grantees supplied the required assur- 
ances in their applications that their projects 
would meet appropriate state/professional 
standards. But to be counted for this table, 
grantees reportedly made more vigorous efforts 
to promote or strengthen standards. 
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Strategy 6; Parent Organization Projects 

Review of Federal Expectations and Assumptions 

In support ins parent organisation projects , the federal program expects 
that grantees will provide technical assistance and information to training 
providers, including parent organisations, that will help parents (and persons 
who assist them) Interact effectively with the system on behalf of their handi- 
capped children. The assumption is that effective interactions will stimulate 
the system to develop and exercise its capacity to meet the needs of handicap- 
ped children for free, appropriate public education and related services. The 
relevant federal program objective is to "expand system capacity." 

According to the relationships presented in Figure 2 in Section I, the 
Personnel Preparation Program expects that providing such technical assistance 
and information will be a major emphasis of grants funded under the Parent 
competition area in Cell V . 

Also included in Cell V is the Personnel Preparation Program's only 
contract, Technical Assistance to Parent Projects (TAPP), through which parent 
projects receive help in designing and implementing their activities from one 
prime contractor and four regional subcontractors. 

Findings 

All projects in Cell V* emphasized the parent organization projects 
strategy, in the ways described below. 

Both the TAPP contractor and subcontractor in the sample report that they : 

Linked parent projects/groups to resources they needed 

Worked with parent projects/groups to assess needs, resources, 
capabilities, and to help plan activities 



* The Parent subsample (H-5> included three grant projects, the TAPP contractor, 
and one of TAPP's regional subcontractors. 
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Emphasized equipping parents to assess needs and to develop 
appropriate interventions, related training, and assistance 

Emphasized y3*r support to sustain, reinforce parents* special 
education role and to motivate parents to increase involvement 

Emphasized networking to encourage exchange of information 

Emphasized identification and dissemination of effective training 
strategies at national and regional conferences, in newsletters 

The srant projects (M-3) in the stu^ qo "T l0 «*ach reported that they: 

Provided parents with information on their handicapped children's 
educational rights 

Provided parents with information on the educ.Monal system's 
responsibilities 

Trained parents to work with educators and others to develop IBPs 
for their children 

Trained parents in effective strategies for working with the 
educational system on behalf of their handicapped children 

Provided "stipends'* (for child care and transportation) to 
increase parent participation 

Used a variety of strategies to recruit parents and volunteers 

Used a variety of strategies to inform the public, school 
systems, and local and state agencies about handicapped 
children's educational rights and about the educational system's 
responsibilities 

Emphasized peer support to sustain, reinforce special educational 
role, and to motivate them to increase their involvement 

Emphasized networking to encourage exchange of information 

Used experts to stimulate, sustain parents' interest and positive 
attitudes 

Project-level data that support implementation of this federal strategy 
include: 

Individualized Technical assistance Plans (ITAPs) 
Records of training and technical assistance provided 
Records of information provided 
Independent evaluation of TAPP contract 
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Monographs, brochures, handbooks produced 
Numbers end types of parent programs developed, improved 
Documented cases of parent interactions with system 
Evaluations of two of the three project's training approaches 

Constraints 

Projects had greatest difficulty reaching minority and rural parents, and 
different strategies were required to attract these target groups. 

Geographical distances for rural parents made transportation to workshops 
an almost insurmountable problem. 

Having only one POP grant in most states, and very few grants in the 
largest states, makes it impossible to address needs of parents in an ade- 
quately comprehensive way. 

Unresolved Issue 

POP grantees differ in opinion as to whether their objective should be to 
train trainers of parents, or to train parents directly. Projects report that 
training trainers is more economical because it requires a less individualized 
approach than is the case in training parents to use effective strategies in 
interacting with the educational system on behalf of their children. 



Strategy 7: System Improvements 

Review of Federal Expectations and assumptions 

The Personnel Preparation Program supports system improvements by funding 
program development and improvement activities that it expects will increase 
the system's ability to meet local, state, and regional needs for trained and 
certified personnel, and for regular educators qualified to educate handicap- 
ped children and youth in least restrictive environments. The assumption is 
that funding such grants will stimulate the system to improve its capacity for 
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personnel development (all levels) in a more timely fashion , and in e more 
comprehensive manner, than would be the case without this extra incentive. 
The relevant federal objective is to "expand system capacity." 

According to the relationships presented in Figure 2 in Section I, the 
Personnel Preparation Program expects that system improvements will be a major 
emphasis of grant activities in Cell IV . that is, the competition areas known 
as Regular Educators and SKA Projects. 

Findings 

All projects in the study sample for Cell IV were judged to be emphasizing 
system improvements . (The abbreviations below appear in the summary table. 
Ms are sample sizes, used in reporting frequencies of findings.) 

Regular Educators (REG, 11=4) 
SEA Projects (SEA, &V4) 

The two most frequently reported system improvements that these projects 
claimed to have promoted Include collaborative planning and Intervention on 
behalf of handicapped children, and preparing trainees who would return to 
their sites to train others to implement model practices. 

Grantees attributed much of their success in enhancing system capacity to 
having been able to involve key institutional decision makers in program 
development, review, or promotion, and to the fact that project participants 
(e.g., regular classroom teachers, special education teachers, parents, build- 
ing administrators) now shared a language, approach, and philosophy that 
facilitated ttieir team efforts. 

Tsble 4 presents information on how these projects implemented the system 
improvements strategy. 
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Table % 

How Grantees Implemented ggfjfesfl Improvements (Strategy 7) 



Number of Projects 
(Cell IV) 

Description REG SEA 

H»4 M-4 



Brophaeized collaboration, teaming as a mechanism for 4 4 

improving capacity of system to assess/address needs of 
handicapped children 

» 

Prepared trainees to provide permanent capacity at their 4 4 

sites, or in their regions, for training others to imple- 
ment program model 



Provided participants with a common language , approach, 4 3 

approach, and philosophy to facilitate team approaches 

Involved key institutional decisionmakers in program 4 1 

development, review, or promotion 



Emphasized developing "partnerships'* in which special 2 1 

educators and regular educators take joint responsibility 
for developing instructional alternatives that can be 
implemented in regular classrooms 



Field-tested state's draft of p re-referral guidelines 2 
in the course of grant activity 

Disseminated statewide training model to other states across 1 
the country 



Prepared building staff (all levels) and parents to assess 1 
needs and to develop appropriate, "locally owned" inter- 
ventions and related training or assistance 




an 



In addition, 29 projects (representing 8 competition areas in Cells X, IX, 
XXI, and V) were judged to be emphasizing system improvements in conducting 
grant activities: 

Special Educators (3/9) 

delated Services (3/5) 

Hural (1/5) 

Transition (4/5) 

Minority (2/3) 

Leadership (5/6) 

Special Projects (6/7) 

Parent Organization Projects (5/5) 

These additional projects reported a wide array of system improvements, 
but the most prominent include those highlighted above: establishing new 
collaborative arrangements and providing permanent training capacity. 

Other prominent examples of the system improvements they reported were 
developing "locally-owned" staff development models, expanding or establishing 
libraries and computer labs, convincing SEAs to support statewide dissemination 
after grants end, and convincing grantee institutions (IHBs) to incorporate 
special education courses or programs developed under the grant in their 
personnel preparation programs. 

Project-level data that support implementation of this federal strategy 
include: 

Self-reports of project staff 
Training materials 

Numbers, types, levels of personnel preparation programs 
developed 
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Constraints 

The two major constraints on implementing system improvements were time 
and money: 

It takes time to get teachers expert enough in a new approach to 
have the ability to fully implement it, and to train others tt 
replicate it. 

Funding cuts are forcing stiff competition for states* special 
education funds, making it difficult to win support for system 
improvements that are tangential to states' current, specific 
priorities (e.g., autistic children). 

The above financial constraint is a special problem for related 
services/roles that are not identified "required." 

Historical turf problems and institutional inertia were also identified 
as constraints on implementing system improvements. 

strategy 8: institutionalization 
Review of Federal expectations and Assumptions 



This strategy was not expected to be a major emphasis of grant activity. 
However, the federal hope is that grant activity will stimulate institutional 
commitments for long-term support for these programs after federal support fo 
them ends. 



Findings 

Some projects in the following competition areas claim to have stimulated 
institutional commitments that appeared to satisfy the criterion of "likely 
long-term support": 



Special Educators (4/9) 
Rural (1/5) 

Regular Educators (3/4) 
SKA Projects (3/4) 
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These long-term commitments took a wide variety of forma, Which did not 
suggest an overall pattern and did not favor a particular competition area or 
cluster of projects. These are examples: 

Expanded scholarship policies to fund students who are committed to 
program area 

Secured XHS support for Interdepartmental training programs 

Obtained vernal assurances from IHB and state to maintain 
undergraduate training program, or incorporate model courses into 
preservice personnel preparation 

Established a permanent statewide training network 

Will soon be rule (with force of law in this state) that regular 
educators must try model's approach before referring a student to 
the special education teacher, and that building administrator is 
accountable for enforcement 

State will support staff to do statewide inservice training in model 
approach 

Project-level data that support implementation of this federal strategy 
include: 

Self-reports of project staff 
Institutional agreements 
Written policies/procedures 

Constraints 

Grantees did not mention specific constraints on implementing Institution- 
alization . However, there are undoubtedly probletmnatic circumstances, which 
could include most of the constraints related to Strategy 7, system improve- 
ments . 
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gjgaag of Findings on Implementation of Federal Strategies 



The following summary recapitulates highlights of findings for the imple- 
mentation of federal strategies through grant activities. 

The Personnel Preparation Program expects to he able to emphasise par- 
ticular federal strategies through grant activities in certain clusters of 
projects. The relationships among program objectives, federal strategies* and 
major types of grant activity were originally shown in Figure 2 (Section I). 
Figure 3 on the next page summarises what the goal evaluation found to be the 
actual implementation pattern (as reported by grantees), using dotted lines to 
show expected primary emphases and solid lines to show actual (multiple) 
emphases . 

As Section I explained, the gross classification that associates particu- 
lar competition areas with one (or two) federal strategies is not intended to 
suggest an exclusive emphasis, but to portray the federal program's strategic 
plan very generally. The Personnel Preparation program fully expects that, in 
aggregate, projects in particular competition areas and cells will implement 
strategies in addition to the one(s) shown as their primary emphasis in 
Figures 2 and 3. 

Projects In the study sample did appear to be implementing strategies in 
addition to the emphasis that was specifically expected for their competition 
area. Table 5 provides a frequency distribution for implementation by the 
study sample of each of the eight federal strategies: by competition area, 
cell, and overall. The frequency distribution in Table 6 shows the number of 
projects emphasising each major type of grant activity that the Personnel 
Preparation Program supports: program development, improvement, and support 
(including stipends); model development, evaluation, and dissemination; and 
technical assistance and information. 

The following summary relates the above Information (in Tables 5 and 6) 
to the relationships originally portrayed in Figure 2 (Section I). 
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Cell I 



Cell I | 
Special Educators (SPED) { 
Related Services (REL) J** 
Rural (RUR) 
Infant (INF) 
Transition (IRA) 
Minority (MIN) 



Recruitment, 
Retention 



Cell III 
Special Projects (SPROJ) 



Cell II 
Leadership (LDR) 



Cell V 
Parent Organization 
Projects (POP) 



Cell IV 
Regular Educators (REG) 
SEA Projects (SEA) 



Targeting 
Critical 
Needs 



Model Devel-i 

optoent , 
Evaluation, 
)isseminationj 



Leadership 
Development 



State and 
Professional 
Standards 



Parent 
Organization 
Projects 



System 
.Improvements 



Institutional- 
ization 



Cell II 



Cell IV 



>A11 Cells 



- Cell 


I 


Pol 1 


TTT 

111 ■ 


^ Cell 


IV 


tf Cell 


II 


. Cell 


I 


L Cell 


II 


* Cell 


III 


5' Cell 


IV 


^ Cell 


V 



All Cells 



Cell I 



Cell IV 



Cell I 
SPED, REL, 
RUR, INF, 
TRA, MIN 



Cell III 
SPROJ 



Cell II 
LDR 



Cell 
POP 


V 




Cell 
REG, 


IV 
SEA 



Key: 



Expected 
emphasis 

, Actual 
emphasis 



Figure 3. Overview of Implementation 
of Eight Federal Strategies 
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Table 5 



Projects Judged to Be Emphasising Particular Federal Strategies 



NOTE: The numbers at the intersections of rows and columns indicate the number of 
projects judged to be emphasizing the strategy in the far left 



Strategy 


SPED 
N-9 


REL 
N-5 


Cell X 
RUR INF 
N-5 N-3 


TRA 
N-5 


KIN 
N-3 


Cell II 
LDR 
N-6 


Cell III 

SPROJ 
N-7 


Cell 

pen 

N-4 


> IV 
N-4 


Cell V 

rUJr 

BK5 


T 
1 

N-30 


Totals by Cell 
II III IV 
N-6 N-7 N-6 


v 

V 

N-5 


overall 
N-56 


1. 


Recrui ttaent/ Retention 


9 


5 


5 


3 


5 


3 


5 






1 




® 


C 
J 




1 




36 


2. 


Targeting Critical 
Needs Areas 


9 


5 


5 


i 


5 


3 


6 


7 


4 


4 


5 




6 


7 


8 


5 


56 


3. 


Model Development, Evalu- 
ation, and Dissemination 


2 




2 










7 


2 


3 








© 


5 




16 


4. 


Leadership Development 














6 












© 








6 


5. 


State/Professional 
Standards 


3 


3 






1 




3 


1 


2 


2 




7 


3 


1 


4 




15 


6* 


Parent Organization 
Projects 






















5 










© 


5 


7. 


System Improvements 


3 


3 


1 




4 


2 


5 


6 


4 


4 


5 


13 


5 


6 


© 


5 


37 


8. 


Inst i t u t ional i 2a t ion 


4 


1 














3 


3 




5 






6 




11 



I 

I 
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Also see Figure 2 



(5) Indicates the part of the Personnel Preparation Program that program logic associates with the strategy in the far left column 
in Section I. 

* The ground rule for judging that a project was "emphasising" a particular strategy was this* Strong elements of the strategy had to be evident from 
both of the following: (a) descriptions of specific efforts or activities that Indicated how the strategy was being forcefully Implemented (provided 
by the interviewee and project documents); and (b) supporting data or information that the project was collecting or was likely to include in its 
final performance report. Examples : 

1. Grantees were not counted as emphasizing model development {Strategy 3) if strong elements of this strategy 
were lacking (e.g. 9 relatively little effort invested in evaluating and revising approaches and materials. 

2. All grantees met the statutory requirement for assurances in grant applications that their projects would 
meet appropriate state/professional standards , but projects were not counted as emphasizing Strategy 5 unless 
they reported vigorous or extraordinary efforts to promote or strengthen state or professional standards. 

3. To be counted for system improvements (Strategy 7) and institutional last ion (Strategy 8) grantees » reported 
that the change or Improvement had moved from the hoped-for or trying-for stage to reality f and had supporting 
evidence. 

Readers who desire more Information on how judgments were made should review the Data Collection and Analysis segment of Section II* and the discus 
slon and definition of each strategy in the present section* 

Projects typically emphasised more than one strategy. Therefore, column totals will not necessarily match Ns for subsamples, and the last column 
will not add to the overall sample size of 56* 
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Table 6 



Projects Judged to Be Emphasizing Major Types of Grant Activity 

NOTE; The numbers at the intersections of rows and coitions indicate the number 
of projects judged to be emphasizing the grant activity in the far left 
coition,* 



l 

I 




(New) Program Development* 
Program Improvement* 

Program Support 

(includes stipends)* 
Expectation : Projects in 
r eils I, II, and IV would 
emphasize one or more of 
the above . 

Model Development, Evalua- 
tion, Dissemination 
Specifically expected of : 
Projects in Cell III 

Technical Assistance and 
Information 

Specifically expected of : 

Projects in Cell V 



3 3 
1 1 



11 



14 



25 



17 
20 
31 



* Projects that emphasized providing stipends may also be counted in the program de lopment or improvement categories, 
will not necessarily match Ns for subsamples, and the last column will not add to the overall sample site of 56. 



Therefore, column totals 
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All projects in competition areas that are specifically expected to con- 
tribute most to the objective of "producing more qualified personnel*' appeared 
to be implementing strategies and activities that the Personnel Preparation 
Program expects are important for achieving this objective: recruitment/ 
retention , and targeting critical needs . To a lesser extent, they also imple- 
mented strategies that program logic associates with the other tiro program 
objectives. These additional strategies included promoting state/profess ional 
standards through their program development and improvement activities, and 
facilitating system improvements . 

All projects that are specifically expected to contribute most to the 
objective of "improving the quality of personnel trained** (i.e., projects in 
the Leadership and Special Projects competition areas) appeared to be imple- 
menting the strategies and activities that the Personnel Preparation Program 
expects will contribute to this objective (i.e., model development, evaluation, 
and dissemination : and leadership development ). In addition, projects in both 
of these competition areas also targeted critical needs and stimulated system 
improvements , and Leadership projects also emphasised recruiting strong candi- 
dates for doctoral and postdoctoral programs, strategies associated with the 
other two program objectives. 

Finally, projects in competition areas associated with the objective of 
"expanding system capacity" appeared to be emphasizing strategies that program 
logic links to this aim. In addition, they targeted critical needs (a strategy 
associated with the first program objective). Projects in the Regular Educa- 
tors and SEA Projects competition areas in addition emphasized model develop- 
ment and state/professional standards , two strategies linked to the third 
program objective. 

In summary, projects in the study sample implemented strategies expected 
to be their primary emphasis and, in addition, emphasized one or more addi- 
tional strategies. Because the sample was representative of the variety of 
grant activities funded by the Personnel Preparation Program, the prospects 
are good for maintaining broad support for these federal strategies among 
grantees . 
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Federal Program Incuts by 
the Divisio n of Peraonnal Preparation (DPP) 



Introduction 

The Personnel Preparation Program Is implemented, in part, through 
federal program inputs by the Division of Personnel Preparation (DPP). DPP 
inputs are pertinent to Column 2 of the Program Logic Model in Figure 1, 
Section X. (The other part of program implementation, strategies pursued 
through grant activities, was discussed above.) 

The analysis of program implementation seeks to answer specific questions. 
These questions are stated at the beginning of the chapter. The questions are 
reworded here for the discussion of DPP inputs: 

To what extent are the intended inputs actually occurring? 

How does this help/hinder federal program objectives? 

What might be done to improve these processes to better support 
federal strategies and objectives? 

The goal evaluation relied on three major data sources for the analysis 
of DPP inputs: 

(1) Two rounds of interviews at the federal level — 

(a) DPP Director, Branch Chiefs, Competition Managers, staff in 
other OSKRS divisions, OSSRS Deputy Assistant Secretary and 
staff* 

(b) DPP Project Officers for projects in the study sample 

(2) Interviews with grantees in the study sample (as consumers of 
DPP support, assistance, leadership, etc.) 

(3) Program documents, including grant announcements for FYS* 
through FYS 7; technical review/evaluation plans for FY84 and 
FY86 application reviews; grant award documentation, including 
justifications for disapproving and approving recommendations of 
peer review panels; telephone monitoring reports in grant files 



* See Appendix S for the list of persons interviewed at the federal level. 




for projects In the study staple; end internal and interagency 
communications (e.g., prominent in the latter category were 
messes to the Grants and Contracts Services I CCS] to expedite 
stalled CCS decisions on grant matters and to forward to DPP 
stalled copies of grantees' final reports). 



Findings are presented under four major headings: (1) Grant Award 
Process; (2) Grant Administration; (3) Leadership; and (4) Coordination/ 
Collaboration. In aggregate, the findings under these headings summarize the 
nature of DPP inputs that were listed in Column 2 of the Program Logic Model 
(see Figure 1. Section I). 

Bach of the four sections addresses the three questions stated above 
under the subheadings of Findings, Constraints, and Recommendations. The 
analysis reflects both federal and field perspectives, and draws from all 
three of the above data sources. 

Grant Award Process 

The grant award process refers to all the activities DPP undertakes to 
provide grants to eligible institutions and organisations for projects in 
priority areas, selected annually for funding in consultation with federal 
officials and representatives of the program's constituencies. These activ- 
ities include setting priorities, announcing priorities and criteria for grant 
competitions, and reviewing and awarding grants. 

Findings . Both federal-level and field personnel agree that overall the 
grant award process is a very fair one. It is the distillation of processes 
and procedures that have been used over the years with successive improvements. 
The process seems to be as good for one competition as for another. Formal 
guidelines for the review process are held to and give credibility to the 
process . 

Beyond any specific process, grantees emphasize the extremely important 
role of federal grant dollars in the success of their programs: grant funds 
have significantly facilitated improvements in and expansion of their programs; 
stimulated cross-department and cross-disciplinary arrangements that have 
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improved institutional capacity; enabled than to take the next logical step 
sooner in incrementally improving their personnel development activities (ell 
levels). 

Funding priorities. Priorities for funding seem to evolve from a 
number of sources. Legitimate pressure comes from Congress, the Secretary* s 
(OSBBS) office, constituencies, even OMB. Some federal staff express serious 
concern that often these priorities are not related to data on critical short- 
ages of personnel, areas of need(s), or an overall, long-range plan. Funding 
levels vary from year to year, and what gets funded appears to many to depend 
on the current "hot issues.'* In focusing funds over the years, the pendulum 
has swung from general to specific priorities and back again. 

Aside from priorities that OPP announces foe competition, many grantees 
in the study sample see funding students (rather than faculty) as the priority. 
They point out that private universities need grants to survive in terms of 
quality , that is, to be able to attract strong candidates is critical. Finan- 
cial assistance allows them to go to school full-time, thus contributing to 
their quality preparation. Grantees commented on the constantly increasing 
cost of going to graduate school (and the cost of living), and hope that 
becoming well qualified will not become a matter of affordability. 

application reviews. DPP must ensure that the best-qualified people 
review the grant applications, and that specific criteria are adhered to in 
selecting applications for funding. A DPP staff member serves as monitor of 
the review panel, seeing that all procedures are followed appropriately. 
Formal contracting and paperwork processing is handled by another unit within 
the Department of Education, Grants and Contracts Services (GCS). 

Grantees comment that the federal application package is clear and easy 
to follow and that it continues to get clearer and more helpful each year. 
They see it as particularly helpful now that the evaluation criteria and 
program guidelines match. (Grantees are experienced grant writers for the 
most part, and some have served on review panels themselves. They note that a 
novice might have difficulty working with the application package. Although 
it is very clear, it is also very long and bureaucratic.) 



-57- 

82 



C onstraints . Many constraints on tha application process are circum- 
stances beyond the control of DPP: GCS, timelines, amount of personnel, 
funding levels. 

CCS.* DPP staff and grantees commonly cite GCS as one of their biggest 
problems in implementing their programs. Mutual concerns about GCS include 
the following: 

GCS is sometimes months late in processing the paperwork after the 
grantee has been notified of the award. Because grantees cannot 
begin work without a written contract, they are delayed in all their 
activities. Recruitment activities (for faculty and students) are 
particularly affected. Starting late may also mean requests for 
budget carryovers at the end of the grant period. 

GCS does not necessarily respond to DPP requests. Grantees, too, 
comment on the numerous telephone calls they make and letters they 
write to get a single response from GCS. 

Timelines. DPP staff and grantees comment on restrictive timelines 
that affect both their activities: 

DPP may be notified late of their funding level and, as a result, 
have very short notice to assemble review panels. Contacting and 
scheduling reviewers requires adequate lead time. 

The timeline between the date of a grant announcement and the dead- 
line date for receipt of applications may be so short that applicants 
j re not able to write about their new ideas. The concern is that an 
inadequate amount of time allotted for the preparation of applica- 
tions hinders creativity and fosters mediocrity. 

Personnel and funding. DPP staff point out that the bottom line is 
the amount of personnel and the amount of dollars available to do the work 
that has to be done. Both DPP and GCS seem clearly understaffed and under- 
funded to do the work expected of them. As a result, **PP spend* most of its 



* For the readers* clarification, the "grants negotiator" is the GCS contact 
for grantees. 
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time "up front" on the grant award process, with little time left for admin- 
istration, leadership, and interagency coordination/collaboration. Grantees 
are understanding and sympathetic with DPP, however, for whom they express 
strong satisfaction in the quality and utility of the contacts they do have, 
although infrequent. 

Other constraints on the grant award process over which DPP may have 
some control inc! do the composition of the review panel and the review of 
critical needs for funding. 

Composition of review panel. Many DPP staff and grantees argue 
strongly for the merits of a review panel composed of both internal and 
external reviewers. The concern is that quality and representativeness may 
be compromised by limiting panels to only internal reviewers, which has been 
proposed as a cost-saving measure. Some grantees have noted a drop in the 
quality of the review process, and have questioned whether reviewers have 
expertise in the areas their applications address. They wonder if onlv 
"insiders" are serving on review panels. Considerable confusion seems to 
exist in the field about who reads applications and how the readers are 
chosen . 

Review of critical needs. Some DPP staff have pointed out that the 
only "needs" information considered in funding grants is the information 
presented in the grant applications themselves. DPP has no means to verify 
this information nor to correlate it with more global, regional needs. Very 
oflen grant applicants reference their state's annual data and information 
in the stated comprehensive plan for personnel development (CSPD) , if this 
is relevant to t a particular training audience under consideration. 
However, information in both sources varies in quality and comprehensiveness 
from state to state. Moreover, states typically do not collect data on 
critical needs for doctoral-level personnel or certain categories of related 
services, nor are state data particularly relevant to parents of handicapped 
children and youth. 
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Overall, the reality is that DPP either gets no data, insufficient 
dat*, conflicting data, or unverlf iable data on "critical needs." Some 
grantees comment that DPP does not have enough money to meet all the needs 
anyway, and, that one critical area is therefore funded at the expense of 
another. 



Recommendations . Recommendations coming from DPP staff and grantees to 
deal with some of the constraints operating on the grant award process 
include the following: 

CCS 

Set up procedures and systems thit expedite grant processing. 

To the extent possible, provide for additional personnel in DPP and 
CCS to alleviate many of the problems. 

Implement joint training for OSEP and CCS staffs with the goal of 
improving operating procedures. 

Inform grantees where they can go for help when GCS is slow and 
unresponsive. 

Timelines 

To the extent possible, coordinate timelines with those of the IHBs: 
— Start the grant award process earlier in the year. 
— Stretch out timelines for continuation awards. 

To the extent possible, (a) release grant announcements earlier in 
order to (b) receive applications earlier (c) so that DPP can select 
the types of field reviewers needed. 

DPP should develop a long-term plan (e.g., a ten -year plan) and 
communicate projections to the field. IHBs need more lead time to 
prepare to be responsive. 

Composition of Review Panel 

Maintain the involvement of external reviewers. It is important for 
accountability purposes to have that expertise and objectivity. 

Provide for a broader base of people in the pool ->f reviewers and a 
quicker system to verify their credentials. 

Make clear to the field how the readers are cuosen, who reads the 
applications, and how reviews are conducted. 
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Review of Critical B»«d« 



Provide DPP with an independent source of reliable information on 
needs. (Developing such a source is beyond any single OSBP 
division's capacity or responsibility.) 

The above recommendations, if acted upon, would strengthen federal inputs 
(by DPP) In ways that would better support the strategies it pursues, through 
the Personnel Preparation Program, to attain program objectives. 



Grant Administration 



Grant administration refers to the system for keeping track of grant 
activity, its aims are to determine what has been produced with the federal 
funds invested in grant projects, increase grantees' sense of accountability, 
and involve DPP staff in coordinating various activities. Grant administra- 
tion also occurs through GCS on contractual and budgetary matters. Although 
administration has many aspects, the focus of discussion in this section is o 
those aspects most commonly cited by federal-level and field personnel: 
ongoing contacts between grantees and federal staff, monitoring, and final 
reporting. 



Findings . The structure of the present grant management system features 
DPP staff serving multiple roles as competition managers, area specialists, 
and project officers who have individual responsibility for all grant activ- 
ities in a specified subset of states. They are guided by decision rules on 
what to monitoi, but each staff mouber has a big project load to manage, and 
accountability problems persist. 

Ongoing contacts. Contacts between DPP and grantees are minimal (two 
or three times a year on the average), but grantees perceive these contacts to 
be of high quality and utility. Aside from the telephone audit that DPP staff 
conduct, grantees typically see themselves as initiating these calls. They 
may call their project officer to determine if they can modify a program idea 
or concept, to make changes in a procedure (e.g., formative evaluation), or, 
more frequently, to find out about upcoming dates for applying to a competi- 
tion, and what future a. sas of funding might be. Occasionally, contact occurs 
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regarding progress on a project. Some grantees point out that if they are 
conducting business in Washington, D.C., they will stop by DPP for general 
project -PR" and to find out about funding plans. 

Grantees are very positive about their contacts with DPP staff. They see 
their project officers as colleagues rather than regulatory in nature, as very 
helpful, very responsive, and always available to answer questions. 

Grantee contacts with GCS are another matter. Budgetary issues ere the 
most common concern of grantees, yet where they perceive that they are likely 
to get the least assistance from federal-level personnel. Grantees cite 
significant difficulty in getting responses from their GCS grants officers 
(e.g., a request for a grant extension took four months for even a response), 
and lost paperwork (e.g., an original and continuation proposal and accompany- 
ing request to transfer the grant to another university). As a result, 
projects suffer setbacks and project staff endure personal hardships. 

Monitoring. DPP staff confirm the minimal contact they have with 
grantees and their inability to keep track adequately of grant activities. 
Civen budget constraints and limited personnel, DPP staff members are able to 
monitor only a small portion of the grant projects and must rely largely on 
faith that grantees are doing what they said they would be doing. The 
monitoring that does occur is carried out by telephone. 

Overall, DPP staff and grantees express a strong need for onsite 
monitoring and point out the valuable opportunities it provides. DPP staff 
gain content and management Information about projects, a sense of their 
reality, and consequently an informed base from which to make professional 
judgments. They gain more exposure, broader experience, and are able to 
provide networking contacts and technical assistance to the field. Grantees, 
in turn, are kept on their toes, gain new Insights about their projects, and 
are able to exchange valuable information with federal-level personnel whom 
they view as colleagues. 

Final reporting and recordkeeping. Final performance reports and other 
official documents on grant projects are filed with CCS. DPP staff point out 
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the difficulty of getting to these files, and an apparent lack of follow- 
through by CCS in forwarding appropriate documents to DPP for evaluation. The 
DPP project officer is responsible for signing off on these projects with 
regard to their completeness and appropriatenass , hut is frustrated by delays 
in receiving final performance reports from GCS and sometimes by not receiving 
them at all. 

Sometimes, grantees themselves have not submitted final performance 
ts. Even those who follow the rules and submit their reports on time 
express doubts as to whether these reports are even read. DPP staff members 
themselves are not sure that the final reports serve any real purpose for 
DPP. Because there are no specific guidelines for 'their preparation, final 
performance reports are submitted in various forms and are neither useful for 
aggregating data nor drawing conclusions. Given the many grant projects they 
administer, project officers are not expected to follow through on these final 
reports. They comment repeatedly that most of their time is spent "up front" 
on the grant award process. 

Constraints . Major constraints on grant administration appear to be 
limited budgets and limited personnel in the two separate federal units that 
are responsible for carrying out the necessary activities. As a result, the 
grant administration that does occur is very limited: 

Although the current types of contacts that occur between DPP and 
grantees are collegial, they are very matter-of-fact and routine. 
They do not affect the quality of programs. 

The current types of contacts that occur between grants officers at 
GCS and grantees are frequently detrimental to project operations 
and to the overall image of GCS in the field. 

DPP staff rely on faith rather than fact that project activities are 
being carried out; this is because it is possible to do only limited 
monitoring by telephone of only a small sample of grantees. 

Under the current system, final reports are seen as of little value 
to DPP staff and grantees. Without specific guidelines for their 
preparation, final reports provide no useful basis for DPP to deter- 
mine some combined effect of federal funding. 

Typically, the proper flow of grantee records from GCS to DPP does 
not occur in a reasonable or timely fashion. 
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Recommendat ions . Recommendations caning from DPP staff and grantees to 
deal with some of the constraints operating on the arant administration 
process include: 

Provide for efficient and economical ways for grantees and federal- 
level personnel to make contact (e.g. , regional meetings of project 
directors and DPP staff). DPP needs to know the people who are 
running the projects, and grantees need to know the people who are 
funding them. 

To the extent possible, provide for at least some onsite monitoring * 
to give DPP staff a sound basis for professional judgment in assess- 
ing the progress of projects. 

Foster communication, coordination, cooperation, and mutual respect 
between DPP and GCS. 

Determine the purpose of final reporting. If final performance 
reports serve only as a culminating rctivity for historical purposes, 
perhaps not much more needs to be done. If they are to serve as the 
basis for drawing conclusions on the overall effects of federal 
funding, then uniform guidelines for their preparation need to be 
established. If they are to be published and disseminated, then 
other guidelines should be followed. 

Provide for centralized grant files with equal access to DPP project 
officers and GCS grants officers. 

Leadership 



Leadership refers to the activities DPP undertakes to guide and shape the 
field of personnel preparation. These activities may include working with 
other units in OSSP to enhance states' responsiveness to CSPD requirements in 
state plans, consulting with the field in planning priorities, providing 
technical assistance to the field in areas of expertise, and disseminating 
ideas and products. 



Findings. DPP engages in a variety of leadership activities in varying 
degrees. Again, most staff time is spent on the grant award process, so 
leadership activities directly related to that receive the most emphasis. 



CSPD. DPP is charged with directing program funds to relieve specific 
needs for various types of special education and related services personnel 
(all levels). DPP attempts to get a sense of these needs from data that 
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states collect annually, from reviewing CSPD portions of state plans for 
special education, and from advice solicited from the field. DPP, through 
its technical review process for evaluating grant applications, is supposed 
to determine whether proposed training projects reflect states 9 data and 
priorities. 

DPP staff have expressed feeling "schizoid" about this determination, 
given that data are not always good or even available for some types of 
personnel. Overall, in fact, these data vary in quality and comprehensive- 
ness from state to state. (See comments under "Review of critical needs, " in 
the discussion for the Grant Award Process, above.) 

It is worth noting that DPP receives only a portion of the CSPD informa- 
tion, and that does not come to them directly from the states, but through the 
Division of Assistance to the States (DAS) , one of five divisions in 0SBP. 
DAS has frontline contact with the states and, from the point of view of DPP 
staff, is in a better position to assist states to improve the quality, 
timeliness, and real utility of their data. 

Some state-level grantees comment that they have only begun to realise 
how their CSPD data might be used more fully. They have so much data that 
their first task is to figure out how best to extract what is relevant to 
particular activities and how best to manipulate the data to highlight needs 
and help set priorities. Such grantees see this as an area where federal 
leadership and technical assistance would be especially valuable. They 
envision DPP working closer with DAS to provide the know-how and guidance 
states need to get the most out of their CSPD efforts. 

Consultation with the field. Federal program managers have tradition- 
ally sought the field* s advice in planning priorities. Appropriately, much of 
the input comes from institutions of higher education (IHSs) because they are 
the knowledge producers, and it is through their programs that new knowledge 
and best practice are communicated to increase the supply and improve the 
quality of trained personnel. 
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Late in the first quarter of calendar year 1986, DPP called in a task 
force of representatives from the field to advise the federal program officials 
on appropriate emphases for the FY87 competitions. DPP has relied heavily on 
task forces as an effective means of consulting with the field. Some staff 
would like to involve the field to an even greater extent, and suggest mechan- 
isms such as task forces and small groups to address specific planning issues. 

Grantees and other field personnel also look to the federal level as a 
central repository for state-of-the-art practice in personnel preparation, and 
for leadership in initiating and facilitating activities in which federal and 
field representatives discuss and develop long-range agendas . * 

Technical assistance. DPP staff answer letters and telephone queries 
in their identified areas of expertise. Grantees are very satisfied with the 
quality and utility of these contacts, although they are infrequent. DPP 
staff and grantees would like more opportunities for technical assistance in 
which they can address substantive issues that affect the quality of programs. 
(Some grantees specifically mentioned that "the old BEH" had provided valuable 
assistance of this type.) In general, the field misses having closer contact 
with DPP staff members. DPP represents the best source of information about 
what is happening nationally—an informal national clearinghouse for ideas, 
innovative practices, staff recruitment, and much mora. 

Dissemination. Without funds for dissemination, DPP does not play such 
a role, except to help grantees submit their products to ERIC for dissemina- 
tion via its computerized database. 

Overall, grantees seem to be unaware of how much or what kind of dissemi- 
nation they should be doing, nor do they have adequate funds in their three- 



* Joint planning has enabled institutions of higher education (IHBs) — to some 
extent— to adjust their programs to better fit fluctuating federal priori- 
ties for personnel preparation. However, to achieve federal goals in a 
lasting, fundamental way, IHBs reportedly need much more lead time — 10 years 
minimum— than a five-year plan or a three-year grant cycle provides. 
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year grants to promote their projects and products extensively. Again, they 
look to the federal level to do something to facilitate dissemination. DPP 
staff members report that they would like to develop hotter dissemination 
approaches and to encourage grantees to place more emphasis on dissemination. 
However, DPP staff are overburdened with work which has higher priority within 
the division. 

Constraints . A familiar constraint on federal program leadership (by 
DPP) is limited resources (funds and personnel) to carry out these various 
activities. Additional constraints in certain areas are noted below. 

CSPD 

DPP's ability to identify critical needs is only as good as readily 
available data and information permit. If and when statutory 
requirements for CSPD are enforced, and each state develops sound 
projections, DPP will have a better (but by no means adequate) basis 
for directing program funds to critical needs. 

Consultation 

DPP project officers are rarely permitted to travel anywhere, so 
consultation must occur by telephone or through task forces in which 
field representatives are brought to Washington, D.C. Task forces 
have proved extremely valuable but are limited in the number of 
field personnel they involve, and the breadth of perspectives they 
can tap. 

Technical Assistance 

Teclinical assistance, for the most part, is limited to answering 
queries by letter or by telephone. Person- to-i-er son contact rarely 
occms, unless grantees are able to visit Washington, D.C. 

Dissemination 

DPP has no dissemination role, and no formal dissemination plan to 
provide to grantees. 

Recommendations . Recommendations of DPP stsff and grantees to help 
overcome the constraints on DPP leadership activities include the following: 

Provide ■ technical assistance to the states in improving the'.r CSPD 
data through a combined DPP/DAS effort. 

Promote the use of task forces and small groups brought to 
Washington, D.C. as a means of consultation with the field. 
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To the extent possible, include technical assistance as part of a 
limited onsite monitoring effort. 

Reinstate a budget for DPP staff participation in professional 
meetings, Which would permit an economical channel for discussion, 
technical assistance, and exchange of ideas. 

Clarify for grantees the dissemination roles of grantees vis & vis 
federal mechanisms or delivery systems; also clarify for grantees 
the underlying logic by which dissemination of promising practices 
will improve the quality of personnel preparation programs, and in 
turn, of personnel trained in those improved programs. 

Consider providing for a national system of sharing "promising" 
ideas, models, programs, and products. The system should be easy to 
access (like a hotline and/or a catalog of brief descriptions and 
contact information). The information available through the system 
should be screened (if not formally validated), and should always be 
current, and easy to translate into practice elsewhere. 

C oordination and Collaboration 



Coordination and collaboration refers to the joint activities DPP unde 
takes with other OSSP divisions, other OSERS offices, and other federal 
agencies associated with personnel preparation. Some of these activities have 
already been mentioned in the above summaries of DPP inputs into federal 
program implementation. 



Findings. Historically, federal agencies have had less than spectacular 
success in implementing truly productive or mutually satisfying and beneficial 
joint arrangements for planning and implementing activities. Members of DPP's 
staff comment on the boxes and boundaries within and without the division that 
prevent integrated programs from being established at both the federal level 
and in the field. Within OSRRS, for example, the needs of special education 
and rehabilitation overlap, yet their representative units — OSBP and RSA — and 
their respective activities remain separate and distinct. Projects in the 
field, as another example, may have to eliminate otherwise appropriate 
elements, such as direct training, in order to "fit" within the boxes and 
boundaries of competition areas (e.g., Special Project grants c&nnot emphasize 
direct training). In short, boxes and boundaries say enhance administrative 
convenience, yet they may also disrupt the integrity of a program. 
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Coordination and collaboration between DPP and GCS presents the most 
immediate and pressing problems for the smooth functioning of the Personnel 
Preparation Program. Both units have experienced staffing changes and budget 
cuts. GCS has undergone two major reorganisations recently, exacerbating 
already present problems in serving DPP and the field efficiently and effec- 
tively. DPP has undergone changes as well, but staff have been able to main- 
tain good relations *Uh the field, as limited as they might be. 

CSPD presents an opportunity for DPP, through coordination and collabora- 
tion with DAS and professional organizations, to determine exactly what states 
are doing and what they might be doing to better address CSPD. Por the most 
part, however, this potential for coordination has yet to be realized. 

Constraints . Obstacles to interagency coordination and collaboration are 
those typical of bureaucratic organisations: each agency operates under a 
separate and distinct legal mandate and funding source, according to 1*4 own 
administrative structure and norms, defending its own turf, and pursuing its 
particular legislative and management objectives. 

Recommendations . DPP staff and field personnel, during interviews with 
the study team, commented frequently that the agencies concerned with personnel 
preparation should be working together to foster communication and cooperation, 
and to define their mutual relet:. Recommendations for overcoming the con- 
straints to coordination and collaboration include: 

Select for these efforts individuals who strongly desire to establish 
cross-agency relationships and who by nature have the persistence 
and organizational development skills to make coordination work. 

Establish formal ties and structured interactions for purposes of 
defining common objectives, for making strategic choices among 
alternative means, and for achieving these objectives. 

Summary, of Findings, on implemantaHon 
of Federal Program Inputs by DPP 

The inputs that DPP is intended to make in implementing the Personnel 
Preparation Program are occurring in various degrees. The grant award 
process , to which DPP staff devote bk-U of their time, is generally occurring 
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as intended. The mechanisms are in place, the processes and procedures that 
have been used over the years have been distilled and improved successively, 
and the formal guidelines for the process are adhered to. A major block to 
the smooth operation of the process is the time delay experienced by many 
grantees in obtaining a formal written contract from GCC* This is particu- 
larly detrimental to grantees' recruitment of both faculty and students, which 
has longer-term impact on the quality of the program. 

The grant admi nistration process (to keep track of grant activity) is 
occurring but only in a "bare-bones" fashion. The structure of the management 
system is in place and seems to be a relatively efficient one, but limitations 
of budget and in number of personnel put DPP staff in a position of having to 
rely primarily on faith that grant activities are occurring as intended, con- 
tacts between DPP staff and grantees, while of adequate quality and utility, 
are loo infrequent to foster discussion of substantive issues that affect the 
quality of grant projects. Comae ts between CCS and grantees on budgetary 
matters are often frustrating. 

DPP leadership activities are occurring In varying degrees, with the most, 
emphasis placed on those related to the grant award process, e.g., consultation 
with the field in planning priorities. The greatest potential for leadership 
occurring would be in working with DAS to enhance states' responsiveness to 
CSPD requirements in state plans. As of PY86, resources were not available to 
DPP for providing any substantive federal leadership through technical assis- 
tance and dissemination activities. 

The potential for coordination and collaboration occurring between DPP 
and other federal agencies concerned with personnel preparation is far greater 
than the reality. Bureaucratic norms are the most serious constraint, but 
short-term fluctuations in priorities and the vagaries of federal program 
funding also make it difficult to initiate coherent and stable collaboration. 



* The "grants negotiator" is the CCS contact for grantees. 
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IV. PROGRAM PERFORMANCE 



Introduction 

A second major purpose of the goal evaluation for the Personnel Prepara- 
tion Program was to determine the extent to which results are beinc achieved 
(by grant projects in the study sample) that support the program goal and 
objectives. 

The goal evaluation methodology has implications for conclusions about 
program performance. First was the decision to draw the sample from "live- 
projects that had had time to stabilize and to implement the strategies of 
interest to the goal evaluation. This necessarily restricted the study to 
conclusions on prospective program performance, based on data that projects 
were presently collecting and were likely to summarize in their final reports 
(generally due at or soon after the end of 1986). 

Second, project reviews in a goal evaluation rely on two major data 
sources: initial and continuation applications in grant files, and interviews 
with project directors or principal investigators, in many cases, grant files 
did not contain the latest continuation, so files were not a useful source of 
information on program performance. Interviews with project directors did 
yield substantial information on project objectives and accomplishments, and 
the nature of supporting evidence that grantees were documenting. 

Third, evaluation resources for this study did not permit data collection 
from third parties, such as consumers. Consumers would have included agencies 
who subsequently utilize the personnel trained and the models or programs 
developed through grant activities. They could have indicated the extent to 
which these products are meeting their critical needs, and are found to be 
high-quality, useful, and effective. 
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Within these caveats, the analysis of program performance was directed at 
answering these questions: 

To which federal objectives did projects perceive they were giving 
most emphasis? 

What results did projects claim they were achieving? 

What evidence were they documenting to support their claims? 

Were their accomplishments consistent with the outputs and outcomes 
the federal program expects from projects? (Reference: The last 
two columns of charts for competition areas in Appendix C.) 

Would outcomes contribute to the three federal program objectives 
and to its ultimate goal? (Reference: The last two columns of the 
logic model in Figure 1, Section I.) 

Federal Objectives That Projects Emphasized 

The relationships among program objectives, federal strategies, and major 
types of grant activity were originally shown in Figure 2 (Section I). 
According to that gross classification scheme, the Personnel Preparation 
Program expects support for its three objectives to come from certain clusters 
of projects, indicated by cell entries. For example, most of the contribution 
to the objective of producing more qualified personnel is expected to come 
from projects in the competition areas in Cell I (Special Educators, Related 
Services, Rural, Infant, Transition, and Minority). In aggregate, projects in 
Cells II and III (Leadership and Special Projects} are specifically expected 
to contribute most to the objective of Improving the quality of personnel; and 
the objective of expanding system capacity is expected to receive most of its 
support from projects in Cells IV and V (Regular Educators projects, SEA 
projects, and Parent projects). 

Associating particular clusters of projects with a particular program 
objective risks conveying the incorrect Impression that "quantity" and 
"quality" are pursued in separate and distinct ways. On the contrary, the 
Personnel Preparation Program strives for parity between quantity/quality. 
For example, the part of the grant program that aims primarily at personnel 
production ("quantity") also has "quality" aspects: 
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• technical review/evaluation procedures are meant to ensure that 
only high-quality personnel preparation programs are funded; and 

• to be funded, applicants must provide assurances that proposed 
programs meet recognized standards . 

The part of the grant program with a pr imary emphasis on "quality" also 
has "quantity" aspects: 

e producing more personnel at doctoral and post-doctoral levels; 

e producing models that address unmet needs for preparing personnel 
in particular specialty areas or for particular roles. 

Figure 4 shows that from a project perspective, too, the drive toward 
federal program objectives was broadly based for grantees in the study sample. 
These grantees, too, perceived that they were contributing to more than on© of 
the three program objectives. Many reported that they were documenting results 
that could be linked to two or all three of the objectives. 

The remainder of this chapter summarizes project accomplishments and the 
nature of supporting evidence to confirm or disconfirm progress toward federal 
program objectives. 



Outputs 

Mature of Outputs and Outcomes 

The charts for competition areas in Appendix C distinguish outputs from 
outcomes by listing them in separate columns. Lists in the outputs columns of 
these charts are more closely akin to processes or activities that occur in 
the course of doing the projects. They typically mirror the statement of 
project objectives in the first column of each chart. Outputs are the focus 
of the present section. 

In contrast, entries in the outcomes columns are more in the domain of 
effects or impacts. Their wording suggests relevance to a particular (federal) 
program objective. Outcomes are the focus of a later section in this chapter. 
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expected to make the major contribution to each federal objective. 
However, both the federal program and its grantees recognize that 
the three objectives are inextricably related, and cannot be 
pursued in separate and distinct ways. 



Figure 4. Overview of Emphasis 
on Federal Program Objectives 



ERIC 



_ , on 

-74- n % t 



I 



Overview of Federal Expectations for Outputs 

Charts for competition areas in Appendix C indicate outputs the federal 
program expected for each competition area according to FYS* grant announce- 
ments. (These charts, with the exception of the c%art for Transition projects, 
originally appeared in the goal evaluation's Program Documentation Report, 
June 1986.) 

Table 7 lists these outputs and, under each one, indicates each competi- 
tion area for which the output is specifically indicated by (1) the charts in 
Appendix C, or (2) the location of the competition area in one of the three 
columns in Figure 2. For example, the Personnel Preparation Program expects 
projects in all competition areas to develop and improve training and related 
information (in the form of programs, models, materials) and to provide this 
training and information directly in all but Special Projects (SPROJ). Another 
predominant output is providing stipends, although only the seven competition 
areas in Cells I and II are specifically expected to do this. 



Findings 

The nature and pattern of outputs that projects in the study sample 
claimed to be achieving was generally consistent with federal expectations foi 
competition areas, and for cells. 

For instance, dissemination of models and materials is specifically 
expected of Special Projects (SPROJ) and SEA Projects (SKA), and all or moot 
projects in these competition areas were documenting evidence to confirm such 
activities. Similarly, all or most projects in competition areas that are 
expected to develop pP Bn ,lwg/mo dels/matflrials ■ to t>rovide training and infor- 
mation , and to offer stipends , were doing so. 

Project-level data that grantees said they were documenting to support 
the above four outputs were predominantly quantitative or concrete: 
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Humbers trained, certified, employed in needs area 
Humbers and content of communications, workshops 
Print and audiovisual products, instruments 
Course descriptions 

Numbers, types, reach of dissemination activities 

For all remaining outputs, supporting evidence is qualitative : 

Opinions of trainees, their supervisors, project staffs, and 
consultants about the quality, utility, strengths, and weaknesses of 
models and materials 

Self -reports (of project staff, participants in training programs) 
of knowledge and competencies acquired, or behaviors changed 

Self -reports (of project staff) of incorporating "best practices," 
identified in research literature, suggested by expert consultants, 
or expected to be effective on the basis of earlier (pilot) efforts 

To summarize, there appears to be good alignment between federal expecta- 
tions and project outputs, and supporting evidence on the extent and nature of 
these achievements. However, available quantitative evidence is likely to be 
limited to production-type outputs, such as individuals trained, models and 
mat-rials developed. Much softer data were reportedly being documented for 
quality-type outputs, such as models evaluated, competencies acquired, best 
practices incorporated. The nature *f these c ita suit the formative emphasis 
of projects' evaluations, which they conducted as part of developing and 
refining their programs and models. 



Outcomes 

Outcomes are the effects or impacts on individuals and organizations of 
training activities, model availability, and "doing the project." Examples of 
outcomes are the availability of more qualified personnel (including regular 
educators); higher quality personnel; new or improved capacity for personnel 
preparation, programming, and planning; interest in or implementation of 
exemplary practices, models, programs. 



o 

ERIC 



-76- 

101 



Table 7 



Outputs Being Documented by Projects In the Goal Evaluation Sample 



Number of Projects That Reportedly Have Supporting Evidence for This Result* 



Type of Output Expected** 


Cell X 

SPED REL RUR IMF TRA MIN 
N-9 N-5 N-5 N-3 N-5 N-3 


Cell 11 
LDR 
N-6 


Cell III 
SPROJ 
N-7 


Cell IV 
REG SEA 
N-4 N-4 


Cell V 
POP 
N-5 


Iwials by Ce.U 
I II III IV V 
N-30 N-6 N-7 N-8 N-5 


Overall 
(N-56) 


Predominant Nature of Grantee's (and TAPP's) Supporting Evidence 


N 


l|IMiUliAifft»AWE MSid 


yuaxitaLAve vara 


• Individuals receive training, informs* 
tion for roles (includes regular edu- 
cators, parents) 

Specifically expected of: SPED, REL, 
RUR, INF, TRA, MIN, LDR, REG, SEA, POP 


9 5 5 3 5 3 


6 




4 A 


5 


30 6 8 5 


49 


• Ns receiving training 

• Ns, content of communications, 
workshops, on-site consultations 




a Programs, models, materials developed/ 
revised 

Specifically expected of: SPED, REL, 
RUR, INF, TRA, MM, LDR, SPROJ, SEA, 
POP 


8 4 5 2 4 2 


4 


7 


4 4 


5 


25 4 7 8 5 


49 


• Print and audio-visual products, 
instruments; lists of competen- 
cies and training objectives; 
course descriptions 




• Stipends attract strong candidates 

Specifically expected of: SPED, REL, 
RUR, INF, TRA, MIN, LDR 


9 5 5 3 4 3 


6 








29 6 


35 


• Ns, types who receive stipends 

• Student records of admissions 
qualifications (for university 
program) 

• Course grades and class stand- 
ing of stipend recipients (for 
university program) 

• Ns of trainees stipends attract 
who would not enter program 
without financial assistance. 


•Letters of recommendation from 
previous employers, administra- 
tors 


• Best practices incorporated in pro- 
grass, models 

bpecit lcaiiy expected ox: mtkuj, 
REG, SEA 






7 


4 3 




7 7 


14 




*i>cii lepoics oi incorporating 
practices based on research liter- 
ature, experts consulted, lessons 
learned from earlier (pilot) efforts. 


• Models and materials evaluated 
Specifically expected of: SPROJ 






7 






7 


7 




•Opinions of trainees, their super- 
visors, project staffs, and consult- 
ants about quality, utility, strengths, 
and weaknesses of model, materials 


* Models and materials disseminated 
Specifically expected of: SPROJ, 
SEA 






5 


4 




5 4 


9 


• Ns, types, and "reach" of model 
dissemination activities, 
consumers 




• Statewide priorities addressed by 
training programs 

Specifically expected of: SEA, REG 








4 4 




8 


8 


• Ns of requests for training 


Hn formal assessments of need 
'Literature review 


• Competencies acquired as a result of 
training 

Specifically expected of: REG, POP 








4 


5 


4 5 


9 




• Self -reports of knowledge, competen- 
cies acquired, behaviors changed 

• Descriptions of parents 9 interactions 
with justice, health, and school 
systems on behalf of their children 



* In order for a project to be counted here, information obtained from the interviewee 
must have indicated that the reported result was being systematically documented and 
that supporting data were being collected or might be reported in final performance 
reports. 

** litis list of "outputs" is consistent with the "outputs" column of the charts for each 
competition area in Appendix C. They represent the federal program's expectations of 
projects funded in FY86. 
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Cell I: Special Educators (SPED) 
Related Services (REL) 
Rural (RUR) 
Infant (INF) 
Transition (TRANS) 
Minority (MIN) 



Competition Areas in Cells I - V: 

Cell II: Leadership (LDR) 
Cell III: Special Projects (SPROJ) 
Cell IV: Regular Educators (REG) 

SEA Projects (SEA) 
Cell V: Parent Organisation 
Projects (POP) 



1(3 



Overview of Federal Expectations for Outcomes 



Charts for competition areas in Appendix C indicate outcomes the Personnel 
Preparation Program expected for each competition area, according to FY86 
grant announcements. Table 8 lists these outcomes (second column) and shows 
their logical relevance to the three federal objectives and to the ultimate 
program goal. 

Findings 

The columns in the middle of Table 8 show the number of projects that 
said they rare documenting supporting evidence for these outcomes. These 
frequencies are presented by competition area» by cell, and for the program as 
a whole. The purpose of this arrangement is to provide a picture that will be 
useful to federal program managers, who will not only be interested in results 
for the program as a whole, but also for units and subunits of the program 
(cells and competition areas). 

If it were possible to lay Table 8, Figure 3 (Section III), and Figure 4 
side by side and to consider cne competition area at a time, and one cell at a 
time, the following picture should emerge: 

Projects in competition areas (and cells) that are expected to make 
the rjst contribution to a given program objective, as a result of 
the strategy(ies) they are expected to emphasize, reported outcomes 
with supporting evidence that were consistent with these expecta- 
tions. 

In addition, support for federal program objectives came from 
projects in other competition areas. 

Having said that, what does Table 8 tell us about the nature of outcomes 
and evidence the study sample claimed to have to support the three federal 
program objectives? 



~1~04 



objective 1 . Forty-four projects represent ins nine competition areas (in 
Cells I, II, and IV) reported that they have quantitative data to show that 
their activities and strategies have produced more qualif ied personnel.* In 
addition, many projects were documenting numbers of program graduates actually 
employed in the specialty areas for which they received training, and some 
were documenting retention data. 

Objective 2 . Eighteen projects representing four competition areas (in 
Cells II, III, and IV) claimed to have improved the quality of personnel 
trained through applying model training approaches, or through doctoral or 
postdoctoral programs. They had only subjective or qualitative data on these 
outcomes, and no objective data on the quality of training or models. Examples 
of supporting evidence were: 

Subjective judgments or ratings of trainees' competency levels as a 
result of training (includes course grades and reports from super- 
visors of practicum experiences) 

Informal feedback (from participants and their supervisors) confirm- 
ing improved performance of individuals trained according to model 

T*ie six projects in Cell II (Leadership) claimed to have increased the 
number of doctoral-level personnel qualified to train teacher trainers, do 
research, and administer programs (i.e., qualified to lead the field of per- 
sonnel preparation) . The data they claimed to be documenting to support this 
outcome included: 

Numbers of graduates (doctoral, postdoctoral) 

Numbers of graduates employed as "leadership'* personnel 



* Projects must now report annually to the Personnel Preparation Program: the 
number of individuals trained under the grant, by category of training and 
level of training; and the number of individuals trained under the grant who 
receive degrees and certification, by category and level of training. 
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Tnblm 8 

Outcomes Seine, Documented by Projects in the Coal Evaluation Sample 
that Contribute to the Federal Frograo Coal and Objective* 



Number of Projects That Reportedly Havt Support teg Evidence for Thle Result* 



Federal Coal and Objective* 



• Obiec* Wo U TO produce 
more qualified perweooel to 
eerre children and youth 
who are handicapped 
Specifically eroected 
ojx SPED* Kit., tUR* 
wr f TtA, HX9 



Related Project Outcomes** 



tncressed quantity of quell fled 
personnel 



S?£0 
K*9 



Cell 

mx aim 

8" 5 t*»3 



S-3 



TRA 
K-5 



H1N 



Cell 11 
1M 
t*»6 



ceil in 

SP80J 



Cell IV 
R£C SEA 
3-6 *«A 



Cell V 
POP 



Tosale bp Ceil 
X SS XXX IV V 
X^30 N»» tr-y N-S K»S 



30 



Overall 



46 



Predominant Hater* of Crmutee's (and TAPP**) Supporting Evidence 



Quantitative Beta 



e *s/ types/ levels of pa raceme 1 
trained and certified U 
specialty or seeds area 

e is of program graduates actually 
employed asd retained la spec lai- 
ty or needs srea for which trained 

e We of trainees demonstrating com- 
petmnclee addraased by training 
program (subjective judgments) 



Qualitative Data 



• Objective 2 ? TO 
the euallcy of personnel 
trained to serve children 
and youth eon are baadl- 
rapped 
Specifically empocted 



o?i ran 



smear 



lap roved quality of personnel 
trained through applying nodal 
crainiag approachae 



Incraaaed number of doctoral- level 
personnel qualified to train 
teacher trainers t do research » ad* 
sinister progrss* (i.e., qualified 
to lead the field of personnel pre- 
paration) 



SOTIj Projects in Cell I 
also aim to lap row the 
quality of personnel 
trained, but are counted 
aa increasing the number 
of qualified personnel. 



4 2 



6 6 



12 



e as of trainees demonstrating con- 
petenciee addreeaed by model or 
training program (subjective 
judgments) 



aa graduating from doctoral (or 
postdoctoral) programs 
a aa of graduates employed ee "lead- 
ership" personnel 



a Objective 3 ; To expand 
the capacity of the 
system for personnel 
development 

Spec 1 fleetly expected 
REG, SEA, POP 



s 



NOTE: Counts are duplicative 
because a project may be 
documenting more than one of 
these outcomes. There are 
actually 43 Individual proj- 
ects (76,7!) represented in 
this array of "syetern capacity* 1 
outcome*. 



new or improved collaborative or 
cross-discipline relet loneMps in- 
crease likelihood of f urth* c joint 
personnel preparation activity, pro- 
gramming , or decisionmaking 

Courses or models made e part of 
eaietlng preserves or doctoral 
programs 

Broad arrey of potential consumers 
indicate serious interest in models 
or programs 

Tfeoie who complete training indicate 
actual or intended use of models or 
training approachae 

Honfederel sources to help promots 
or sxpend demons t ra t 1 on /dissemina- 
tion of model 

Training (includes pa rente) creates 
new, permanent, local capacity for 
personnel preparation 



Parents (or persons who assist then) 
engage in educational planning and 
progrsmmlng with the system on behalf 
of their children 

Raw statewide training network aa a 
result of project 



3 2 



3 3 



4 3 



11 14 7 1 



24 



2 2 



1 12 11 



1 1 1 



2 1 



2 3 



4 2 



7 3 



6 4 



4 5 



3 6 5 



16 



11 



13 



IS 



3 4 



e Ultimata goal ? Enhance edu- 
cation and related services 
for handicapped children and 
youth through the prepara- 
tion of specialised person 
eel 



educational or social attainments 
improve for children and youth 
who are handicapped 

Completers of training provide 
improved quantity, quality of 
services to children and youth 
mho are handicapped 



6 3 5 



3 2 2 2 3 1 



3 1 



13 6 4 1 



IS 6 



26 



25 



a ma and types ef changes in class- 
room, LEA, SEA, or XBC proced- 
ures, pollclss, programs 
Written institutional agreement ■ 



a Na of INEs Incorporating training 
model or program in curriculum 

e He of (written) requests for in for- 
est ion, training 



lie end nature of implementation 
(subjective j ud finaots of level 
of implsmentstion) 



e Ha of trainees equipped to provide 
training in model or program 
approach to personnel at their 
local eitea (subjsctlve judg- 
ments ) 

a Number of hours spent facilitating 
parent involvement in education* 
el planning and programming 



a geeulte of evaluation atudiea 



Sslf -reports of trainees* mastery 
of competencies or behaviore 
addressed by the training program 

Informal feedback c< nf liming Im- 
proved performance of individ- 
uals trained according to model 



• Self-reports of project staff 
a Informal feedback from partici- 
pants 



a Self-reporte of project staCf, 
trainees 

e Informal feedback from tralneee 

a Self-reporte of project eteff 
Agreements (not necessarily 

written) 

• Self-reporte of project eteff 



e Follow-up questionnaires to 

parents (eelf -reports) 
s Informal feedback from parents 



e Self-reports of project stsff 



e Self-reporte of project eteff 
Feedback from supervisors 



s Informal feedback from employers 
e Self-reporte of trolnees, aemlele- 
tretore et practlcis* sites, 
graduates of doc torsi programs 



• In order for e project to be counted here, information obtained from the interviewee must have indicated that the reported result 
was being systematically documented sm- that supporting dets were being collected or might be reported in final performance reports. 
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the "related project outcomes'* (Column 2) for Object tves 1 sad 2 follow the wording in the "outcome*** column of the chsrts for 
competition areas in Append is C. For Objective 3 . the outcomes 'ill not be found in the chart* in Appendix C, but do reflect 
language in the FYS6 grant announcements for sever si of these competition erase. Overall • the outcomes in Column 2 above 
H federel empec rations of projects funded for Flea. 



Competition Areas in Colin I - V* 



Call I: Special Educators (SPED) Cell XX: 

ftslatad Services (BEL) Cell III; 

sural (act) Cell IV; 
Infant (1*T) 

Treneitloa (TftnJfS) Cell V: 
Hinority (mo 



lesdershlp (UMt) 
Special Projects (5PX0J) 
Regular Educators (REC) 
SEA Projects (SEA) 
Parent Or gaol est ions 
Projects (POP) 



117 



Objective 3 . Forty-three projects, representing all competition areas, 
claimed to be achieving outcomes that were relevant to expand inn system 
capacity for personnel development (all levels). Frequencies in Table 8 
suggest that the most prevalent of these outcomes Included: 

New or improved collaborative or cross-discipline relationships that 
increase the likelihood of further joint personnel preparation 
activity, programming, or decision making 

New, permanent, local capacity for personnel preparation as a result 
of project activity 

Serious interest in projects' products (models or programs) across a 
broad array of potential consumers 

Commitments of nonfederal support to help promote or expand demon- 
stration and dissemination of models, programs 

These projects said they were documenting quantitative data to support 
most of their claims. However, this information is likely to be imbedded in a 
descriptive narrative in their final reports, rather than in a format that 
would make it easy to extract and aggregate across projects and competition 
areas. 

The ultimate goal : Nearly half of the projects in (Cells I, II, III, and 
IV) planned to report qualitative data to support their claims of: 

Improved educational or social attainments of children anl. youth who 
are handicapped 

Improved quantity and quality of services to handicapped children 
and youth (from participants who complete training) 

One project in Cell IV (in the Regular Educators competition area) plan- 
ned to collect quantitative data that he expected would show a reduction in 
the frequency with which classroom teachers referred students for special 
education services. (This project has developed a program in which regular 
educators and special education personnel jointly develop instructional stra- 
tegies for teaching learning disabled students in regular classrooms. The 
classroom teachers "commit" to trying these instructional alternatives first, 
before resorting to referral for special education services, and building 
administrators "commit" to ensuring that they do.) 

" 8 t08 



""ffffmf nf Findings on Program Performance 



nearly 80% of the study sample (44 projects in 9 competition areas) 
claimed to be achieving outcomes and to have supporting evidence for the 
objective of "producing store qualified personnel." Over 30% (18 projects 
in 4 competition aress) said they were achieving outcomes associated with 
the objective of "improving the quality of personnel trained." Over 75% 
(43 projects in all 11 competition areas) reported outcomes that represented 
the third objective, "expanding system capacity." Most prevalent were new or 
improved collaborative arrangements as a result of grant activity, claimed by 
about 43% (24 projects in 9 competition areas). 

Therefore, there appears to be broad support for federal program objec- 
tives from results that grantees in the study sample were trying to achieve. 
This suggests that the prospects for satisfactory program performance are 
positive. 

However, it is likely that the only quantitative data available in 
summary form will be relevant to producing more qualified personnel and more 
"leadership" personnel. The data to support the objective of "improving the 
quality of personnel trained" will be quite soft, and quantitative data to 
support the "system capacity" objective, though impressive, will probably not 
be provided in a form that makes it economical to aggregate. 

Finally, there was a surprising amount of interest among grantees in 
tracking ultimate benefits for handicapped students, given the impossibility 
of conclusively demonstrating that these benefits are attributable (directly) 
to personnel preparation programs. Hearty half of the projects in the study 
sample claimed to have information on improved-service delivery to, or improved 
attainments of, handicapped students who were subsequently served by project 
participants. 
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V. PROGRAM PLAUSIBILITY 



Introduction 

This step in the goal evaluation asks the question: 

"Is it reasonable to expect the Personnel Preparation Program to 
achieve its objectives, given the federal strategies that have been 
adopted, and the extent to Which they are being supported by the 
program and implemented through grant activities that are actually 
operating in the field*" 

This question reflects what the t*rm, "plausibility," means in the context 
of a goal evaluation, and the nature of judgments that the evaluator makes. 
An objective is plausible if there is some likelihood that program activities 
and strategies will achieve progress toward the objective. To judge an objec- 
tive as plausible would be to claim that if the program continued to operate 
as observed, the objective would or could be attained. Saying that an objec- 
tive was implausible would be to assert that, for various reasons (which the 
evaluator makes explicit), the program operations or conditions observed do 
not support attainment of the objective. These reasons could include such 
factors as the failure of activities or strategies supporting the objective to 
occur as planned. 

Procedure 

Making these judgments requires estimating the extent to which the inputs . 
processes , and assumptions represented in the program logic model are occurring 
or can occur at a level of performance that justifies expectations about the 
Personnel Preparation Program's ability to achieve its objectives. 

Much of the information needed for the plausiblity analysis is in 
Section III (Progt . Implementation): 

Findings on the extent to which processes and Inputs are being 
implemented (i.e., strategies, project and federal program 
activities) 
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Conclusions about the congruity or incongruity between expected and 
actual enchases on federal stratesles through major kinds of grant 
activity. 

Information in Section IV (Program Performance) is also relevant: 

The nature of results grantees claimed to be achieving, and the 
extent to which these results appear to support Personnel 
Preparation Program objectives 

All of this information must now be reexamined in making judgments about 
program plausibility. 

The analysis addresses these questions for each of the three program 
objectives: 

To what extent were federal strategies being implemented through 
project activities? (Ref : Section III) 

To what extent were projects' objectives and accomplishments 
congruent with federal program goals and assumptions? (Ref: 
Section IV) 

To what extent does evidence of these accomplishments exist? 
(Ref: Section IV) 

Methodological Reminders 

The goal evaluation examines strategies as they operated through FY86. 
This also establishes the time boundary for the plausibility analysis. While 
it is conceivable that changes from FYS 6 to FY87 have implications for program 
plausibility in the future, the present analysis must necessarily be restric- 
ted to the data base for the goal evaluation. 

It is not within the scope of a goal evaluation to collect primary data 
on project accomplishments, to capture all relevant perspectives, or to verify 
secondary data conclusively. The study team relies on readily available writ- 
ten or verbal reports of results and, to an even greater extent, on interviews 
with principal investigators or project directors— who are certainly not 
disinterested observers. Even though this study team conducted interviews on 
a confidential basis, and most interviewees were candid ("This is off the 
record. . ."), it is certainly posslole that some relevant information is 
fugitive. 
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The goal evaluation sample is small in proportion to the sire of the 
program, although it is representative of the broad array of Personnel Prepa- 
ration Program grant activities and six of the eleven suhsamples constituted 
between 25% and 37% of their sampling pools, another compensating advantage 
for a program analysis of this type is that data collection emphasizes depth, 
especially in the areas that will inform a plausibility analysis. 

Finally, goal evaluations do not examine program management procedures 
per ge. However, it is legitimate (and necessary) to determine if intended 
major program inputs are occurring at a level that supports program objective 
and federal strategies that are implemented to attain the objectives. This 
information, too, is part of the database for the plausibility analysis. 

Organization of Presentation 

The plausibility analysis for each program objective is divided into 
three sections: (1) analysis; (2) constraints and unresolved Issues; and (3) 
summary. The analysis addresses the three questions (above) that are at the 
heart of a plausibility analysis. The list of constraints and unresolved 
issues includes a variety of perspectives: grantees; DPP, OSBP, and OSERS 
staff; Personnel Preparation Program constituencies in the field; and 
literature reviewed in the course of the goal evaluation. Some of these 
issues were mentioned in the Program Documentation Report for the goal 
evaluation. They reemerge here because they affect strategies as they 
currently operate, and because they could affect the future plausibility 
of program objectives. 

Plausibility of Objective 1 

This federal objective is "to produce more qualified personnel to serve 
handicapped children and youth." 



Judgment : At face value, Objective 1 is plausible, 
as it is presently stated. 



ERIC 



-87- 

112 



Analysis 



As of FY86, there were six competitions directed at filling shortages and 
targeting for critical areas of need: Special Educators. Related Services, 
Rural, Infant, Transition, and Minority. 

Section III documented that all projects in these six competition arear 
appeared to be implementing the strategies and activities that the Personnel 
Preparation Program expects are important for achieving Objective 1. They 
said that they were documenting numbers and qualifications of recruits , and 
several said that they had retention data. Their proposals presented convinc- 
ing statements of critical needs that incorporated or cited a variety of 
appropriate sources, and comments of peer review panelists were particularly 
complimentary on this aspect of grant applications. 

Section IV indicated that the training program? and materials that these 
projects subsequently developed, and the training they provided, were directed 
at the needs they proposed to address, and appeared to be quite consistent 
with the focus of the competition areas under which they were funded. Project 
level data reportedly includeo umbers trained and certified, by specialty 
area and/or degree level, and numbers of training "graduates" who enter the 
roles for which they were trai led. (With the exception of Leadership projects, 
grantees did not systematical y collect data to show whether graduates of 
training progress remained in careers in special education.) In summary , 
these data supported outputs and outcomes that the federal program expects of 
these types of projects. 

The above picture also holds for additional subeamples (1) that imple- 
mented these two federal strategies, and/or (2) whose major grant activity was 
"program development, improvement, and support (including stipends), even 
though Objective 1 was not necessarily their primary emphasis. For example, 
all projects in all other competition areas in the study sample claimed to be 
targeting critical needs, and all but one of the projects in the Leadership 
subsample emphasized recruitment . They all claimed to have quantitative data 
like that noted above for other subsamples. 
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Constraints and Unresolved Issues 



The lists of constraints and issues below pertain, In turn, to 
Strategy 1, Strategy 2, and Objective 1. 



Strategy 1. Recruitment and Retention 

Stipend allowances are not always a sufficient incentive for 
qualified candidates with financially attractive options. This was 
especially true for strong minority candidates, who appear to have 
many options. 

For projects in the Leadership competition area, high tuition and 
selectivity constrained how many candidates doctoral and post- 
doctoral programs could recruit. 

The methodological difficulties and costs of longitudinal surveys 
make it unlikely th.it grantees will ever be a feasible source of 
reliable data on whether those who prepare for and enter careers in 
special education remain in these careers. 

Strategy 2. Targeting Critical Meeds Areas 

The intransigent problem of what constitutes a shortage, how to 
identify it, and how to focus resources on relieving it is far from 
being resolved (or completely understood). 

While the larger strategic issues in targeting critical needs areas 
are receiving varying amounts of attention from OSSP, OSGRS, and 
agencies outside OSSSS, such as OPBE and the Center for Statistics, 
DPP has not yet assumed a significant role, or developed a long-range 
plan for sharing in the problem's resolution. (The FY86 amendments 
establish a national clearinghouse whose responsibilities include 
collecting data on needs. If DPP can influence or shape data 
collection so that it includes more than "teacher counts," that 
represents an opportunity to make some headway.) 

Ho satisfactory system or mechanism is functioning well enough at 
present to provide a baseline against which to evaluate grantees' 
documentation of critical needs areas in initial and continuation 
applications. 

CSPD information and states' annual counts are least useful for 
documenting needs for highly specific personnel specialities, and 
for projects serving regional and national interests or needs (such 
as those submitted under the Leadership competition). 
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There are no central data bases relevant to preparing personnel to 
serve handicapped individuals in out-of -school settings (e.g., 
hospitals). 

Uncompetitive salaries and unappealing geographical location are two 
problems (not amenable to training program Interventions) in target- 
ing critical needs areas. 

Because of critical shortages, uncertified teachers are hired, and 
this reduces the incentive for potential trainees to enter certifi- 
cation programs. 

Objective 1 ; 

Grantees observed that it takes a long time for individuals who 
receive training in model practices to develop "working power" 
(competence) at a level necessary for a meaningful head count of 
qualified personnel. It also takes time to be qualified for 
providing effective training to others. 

Mo universal standard exists to suggest what "qualified" means. 
State certification requirements vary widely. Grantees' data on 
competencies demonstrated as a result of training were also weak, 
relying heavily on subjective assessments of trainees* performance 
by their teachers, practlcuro supervisors, and employers. 

Grantees took issue with being associated (in federal program logic) 
with personnel production. They counted among their most significant 
accomplishments having been able to win commitments and engineer 
changes in their institutional contexts that would continue the pro- 
grams they had established. Such improved capacity, these grantees 
reasoned, will eventually enable the system to meet its needs for 
(sufficient numbers of) qualified personnel. 

Summary 



This analysis concludes that: 

There is quantitative data that can be aggregated to indicate the 
numbers trained, for what specialty areas, and at what degree 
levels.* Grantees also have data on numbers employed in roles for 
which they were trained. All of this information will provide 
support for the "personnel production" aspect of Objective 1. 



* Projects must report annually to the Personnel Preparation Program: the number 
of individuals trained under the grant, by category of training and level of 
training; and the number of individuals trained inder the grant who receive 
degrees and certification, by category and level of training. Grantees enter 
the data on a form that DPP refers to (in-bouse) as "the 98-199 form," because 
the amendments to the Education of the Handicapped Act in this public law 
authorized such data collection for the first time from Personnel Preparation 
Program projects. 

-90- 
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According to the statutory language, "qualified" means that the person 
who completes training also meets state requirements or professionally 
recognised standard* for a certificate or degree. Thus, grantees* data 
on the numbers who earn certificates or degrees In specialty areas will 
provide support for the Qualified portion of Objective 1. 

Strategies 1 and 2, recruitment end retention and targeting cr^cal 
needs areas , can be implemented as intended through program development, 
improvement, and support grant activities. 

There was a very close fit between (1) critical needs areas documented in 
grantees' proposals and (2) programs and models they developed, or 
training they delivered. 

It would be difficult to convince individual grantees in the "personnel 
production" competition areas (SPED, RBL, RUB, IBP, TRA, Mill) that the 
primary importance of their projects was to increase the quantity of 
qualified personnel. From their perspectives (orolect level), this 
objective is a much more distal outcome, following from improved system 
capacity . 



Plausibility of Objective 2 

This federal program objective is "to improve the quality of personnel 
trained to serve handicapped children and youth." 



Judgment: In the absence of objective assess- 
ment, the plausibility of Objective 2 is neither 
confirmed nor disconf irmed. 



Analysis 

According to the gross classification presented in Figure 2 and explained 
in Section 1, there are two competitions directed primarily at improving the 
quality of personnel preparation: Special Projects and Leadership. These 
grants support, respectively, (1) development of exemplary models for person- 
nel preparation, and (2) preparation, at the doctoral and postdoctoral levels, 
of individuals who will then so on to encourage the use of state-of-the-art 
methods in personnel preparation. The Personnel Preparation Program would 
like to be able to show that benefits of these grant activities address 
critical needs for "leadership" personnel and for replicable, exportable 
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training models and cost-effective approaches to personnel development (all 
levels). 



Section III concluded that projects in these two competition areas appeared 

to be Implementing strategies and activities that the Personnel Preparation 

Program expects to be their primary emphasis, and to be associated with the 
^ecor. i objective: 

All Special Projects in the study sample undertook activities that 
included strong elements of model development, evaluation, and 
dissemination (Strategy 3) . 

Leadership projects emphasized leadership development (Strategy 4) , 
through doctoral and postdoctoral programs they developed, improved, 
or supported with grants from the Personnel Preparation Program. 

Two important qualifications pertain to Strategy 3, as it was implemented 
by projects in the study sample. First, model evaluation relied on expert 
review and participant feedback, carried out in the context of the model's 
developmental try out; formal field tests did not occur. Second, model 
dissemination emphasized one-way communication — getting information out to 
target markets, and promoting model approaches through presentations at 
conferences and workshops. Although a wide variety of potential adopters 
expressed serious interest in the model approaches, there wa.t little oppor- 
tunity within a three-year grant period to provide them with implementation 
assistance, nor was this expected to be an objective of such grants. 

Section IV described the data that Special Projects and Leadership 
grantees said they were documenting of outputs and outcomes that contribute to 
the quality objective. Their quantitative, objective data was reportedly 
limited to product ion- type outputs: 

(For Special Projects) numbers of models and materials developed, 
target rales and levels, and the frequency, nature and reach of 
dissemination activities 

(For Leadership projects) numbers of doctoral and postdoctoral 
graduates, numbers of graduates employed as "leadership* 4 personnel* 



* The definition of "leadership** roles varies, as does the availability of 
such opportunities for new Ph.Ds. 
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Ho objective data are likely to be available on the quality of models* 
the quality of training* or improved quality of personnel trained through CI) 
applying model training approaches or through (2) doctoral or postdoctoral 
programs. However* these are examples of the qualitative data they said they 
were documenting: subjective judgments or ratings of competency levels that 
trainees demonstrated during and upon completion of training (Including course 
grades and reports from supervisors of practlcum experiences); and informal 
feedback from participants and their supervisors confirming performance 
Improvements and desired changes in behaviors and attitudes of individuals 
trained. 

It was beyond the scope of grant projects to obtain data from consumers 
of the models they developed and the personnel they trained. Such data could 
indicate whether these products in fact do meet critical needs of the field, 
and are found to be high-quality, useful* and effective. 

Constraints and Unresolved Issues 

The lists of constraints and issues below pertain* in turn, to Strategy 3, 
Strategy 4, and Objective 2. 

Strategy 3. Model Development. Evaluation. Dissemination 

Grant budgets and timelines are not sufficient for carefully testing 
models, or for distributing them and providing implementation 
assistance on a large scale. 

Federal program resources have not been used to fill this gap* 
except selectively. For example, funds were added to a grant that 
developed a training system for infant service providers, to enablr 
the model developer to do a workshop for all Personnel Preparation 
Program grantees with "infant" projects. 

Strategy 4. Leadership Development 

Budget reductions at institutions of higher education, combined with 
the relatively high cost of doctoral programs and limited availabil- 
ity of stipends to attract well qualified full-time doctoral stu- 
dents, threaten tho quality of future leadership for the field of 
personnel preparation. This dilemma results from economic condi- 
tions that the Personnel Preparation Program cannot affect. 
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Opportunities for new Ph.Ds to assume leadership roles have been 
limited to date, a situation that is not amenable to training 
Interventions. 



Objective 2; 

Quality-oriented intiatives hav<* inherent measurement problems that 
make it very difficult and costly for grantees to establish 
rigorously the merit of their grant products (models, leadership 
personnel). Not surprisingly* available quantitative, objective 
data for adequately assessing accomplishment of this objective are 
virtually nil. 

Even qualitative data (e.g. 9 subjective assessments of demonstrated 
competencies, of model quality, of behaviors acquired through 
training that utilizes state-of-the-art practices) have not been 
obtained from disinterested sources, especially consumers.* 

Summary 



This analysis concludes that: 

Assumptions that these two federal strategies will lead to improved 
practice and to improved quality of personnel are neither confirmed 
nor disconfirmed on the basis of data available from the study sample. 

As grant activities are presently focused, it is probably not 
feasible to expect grantees to obtain data that would rigorously 
support Objective 2. 

Strategy 4, leadership development , can be implemented as intended 
through doctoral and post-doctoral grant activities. 



* By definition, a disinterested data source must be limited to individuals 
who were not associated in any capacity with the grant project, because 
otherwise their roles would compromise their absolute objectivity. To be 
included in such a survey, consumers oust actually have "purchased" the 
services of the graduates of doctoral programs, or "purchased" (and actually 
implemented) models or materials that were developed and refined during 
grant projects. By this definition, consumers exclude experts who were 
involved in a model's formative evaluation (developmental tryout), or 
individuals who have roles in the programs Whose graduates are being 
evaluated. 
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Strategy 3. modal development, evaluation, and dissemination, can be 
implemented to e limited extent within a three-year grant » with most 
of the effort going to model development 9 promoting awareness of 
these models, and exposing others to the models during their 
developmental tryout. 

There was a very good fit between models developed by grantees and 
the critical needs that they documented in their grant applications, 
and there was evidence of serious interest among potential consumers. 
Thus, the models have the potential for meeting critical needs. 

As grant activities are presently focused, it is probably not 
feasible to expect granteeg to obtain data that would rigorously 
support Objective 2. 

Plausibility of Objective 3 

This federal objective is "to expand the capacity of the system for 
personnel development." 



Judgment : Objective 3 is plausible* 



Analysis 

As of FY86 , there were three "capacity-oriented" competitions: Regular 
Educators, SEA Projects, and Parent projects* The first two competitions were 
directed at stimulating improvements in the system's ability to meet local, 
state, and regional needs for trained and certified personnel, and for regular 
educators qualified to educate handicapped children and youth in least restric- 
tive environments. Parent projects were to provide training, technical assis- 
tance, and information to parents to help them influence the system to develop 
and exercise its capacity to meet the needs of their handicapped children. 

Section III documented that all projects in these three competition areas 
appeared to be implementing the strategies and activities that the Personnel 
Preparation Program expects are important for achieving Objective 3. Overall, 
they developed programs and provided technical assistance, information, and 
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training that emphasized collaborative planning and intervention on behalf of 
handicapped children, and permanent local or statewide training capacity to 
stimulate replication of their models and approaches. 

Sections III and IV indicated that these projects said they were docu- 
menting quantitative data on the number and nature of their Information 
exchanges, training events, and technical assistance sessions; the amber and 
nature of parent and personnel preparation programs they developed; the 
number, nature, and reach of their dissemination efforts; and the number and 
location of individuals, agencies, and groups Who express serious interest in 
the training approaches they have developed. To some extent, these projects 
were documenting new collaborative or cross-discipline planning mechanisms; 
new, permanent, local capacity for personnel preparation; and commitments of 
non-federal support for model demonstration and dissemination. 

It is significant that some projects in every one of the additional eight 
competition areas also claimed to have evidence of stimulating system improve- 
ments . In fact, in every case where a grantee could point to supporting data 
for having brought about or contributed to such improvements, they counted 
these accomplishments among the most important results of their grant projects, 
even if they were not in competition areas that were expected to contribute to 
Objective 3. 

Constraints and Unresolved Issues 

The lists of constraints and issues below pertain, in turn, to Strategy 6, 
Strategy 7, and Objective 3. 

Strategy 6. Parent Organization Projects 

Having only one POP grant in each state makes it impossible to meet 
needs of parents in large or populous states. 

A persistent implementation problem has been reaching minority and 
rural parents. 

Grantees* opinions differed on whether their objective should he to 
train trainers of parents, or to train parents directly. The latter 
requires an Individualised approach, and is therefore more expensive. 
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Strategy 7. System Tmpi^wiwiynta 



Time, money, and institutional inertia were the major constraints for 
bringing about system improvements: time to acquire the proficiency 
required of a "master** trainer who teaches others to replicate model 
practices; Intense competition for states* special education funds; 
and resistance to changing established organisational behavior and 
structure. 

Objective 3: 

The Personnel Preparation Program presently lacks a composite picture 
(synthesized from a variety of sources) to use as a basis for deci- 
sions to maintain or to expand system capacity through the grant 
process. Without this information, it is quite conceivable to curtail 
system capacity unwittingly, as is the case when a decision not to 
support a grant disables a state from taking the next logical step in 
improving its personnel development program, or from undertaking 
activities that would otherwise take many more years to accomplish, 
given other state priorities and limited resources. 



Summary 



This analysis concludes that: 

Strategies 6 and 7, parent organisation projects and system improve- 
ments, can be implemented as intended through grant activities that 
are associated with Objective 3. 

Data are available to indicate the number and nature of a wide 
variety of system improvements (including those involving parents 
and parent organizations). However, much of this information is 
likely to be threaded through narrative sections of grantees' final 
reports, making it difficult to extract and aggregate. 

There was a very close fit among (1) critical needs areas documented 
in grant applications; (2) the programs, models, materials, and 
approaches that grantees developed; and (3) the training and assis- 
tance they delivered. 
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Summary of Findings on Pro«ram Plausibility 



This section closes with the observation thai; none of the program objec- 
tives was strictly implausible. Moreover, support for these objectives through 
grant activities among projects in the study sample was much broader based than 
might be assumed from the primary emphases portrayed in Ft >re 2. Grant proj- 
ects Implemented strategies and pursued objectives in addition to those expected 
to be their primary emphasis. (This is no surprise to DPP staff or to the work 
group, who intended Figure 2 to provide a gross classification, useful for 
focusing data collection and analysis in the goal evaluation, and for indicating 
the program's overall strategic plan.) 
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VI. CONCLUSIONS AND RECOMMENDATIONS 



Introduction 

Section I of this report described prop ram logic through figures and text 
that explained the federal strategies being implemen v .ed through the grant 
program to reach program objectives, and assumptions about how these actions 
are expected to contribute to these objectives. 

Sections ill and IV analysed information from 56 project reviews as a 
basis for assessing the validity of the logic and assumptions underlying the 
Personnel Preparation Program (as it operated in FY86>, and for judging the 
extent to which the strategies are being implemented and results are being 
achieved that support federal program objectives. 

Section V reexamined this information to draw conclusions about the 
reasonableness of expecting the Personnel Preparation Program to achieve its 
stated objectives if strategies continue to operate as observed in the study 
sample, with similar results and supporting data. 

The generally positive findings In the preceding chapters support the 
conclusions that follow, but also indicate areas that could profit from 
further examination. 
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Conclusions* 



Strategies Can Be Igaalamented Through Grant Activities 
To An Bctent That Supp orts Program Objectives 

and 

Project Results Support Program Objectives 

All projects in the sample were judged to be (1) implementing the federal 
strategies that were expected to be their primary emphasis and, in addition, 
(2) one or more of the strategies associated with other program objectives 
(and competition foci). Overall, the nature of quantitative and qualitative 
evidence of grantees' activities and accomplishments, provided elsewhere in 
this report, Indicates a good fit with federal expectations and with program 
objectives. 

Therefore, prospects are positive that the Personnel Preparation Program, 
by pursuing federal strategies through its grant programs, can make progress 
toward Increasing the number of qualified personnel to serve handicapped 
children and *outh, improving their quality, and expanding the capacity of the 
system for personnel development. 

Many Project Results Are Well Documented 

Nearly 80% of the study sample (representing nine competition areas) 
claimed to be achieving outcomes, backed up with quantitative data, that 
pertain to the first program objective, "to produce more qualified personnel." 
The most readily available data included: numbers of individuals recruited, 
trained, and graduated (by level and specialty); number of program graduates 
who subsequently enter careers in special education in roles and areas for 
which they were trained; number and nature of grantees* training, technical 
assistance, and dissemination activities; and the number and nature of the 
models and materials they developed. 



* Section II described the goal evaluation methodology, and Sections IV and V 
reiterated its implications for interpreting findings and conclusions. 
These caveats are not repeated here. 
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Over 30% of the study sample (representing four competition areas) 
reported outcomes and claimed to have data to support the second progam 
objective, "to improve the quality of personnel trained.** These data, 
however, are subjective and qualitative. For example, data that grantees in 
the study sample were documenting on model quality, improved competence, and 
use of state-of-the-art practice in personnel preparation consisted mostly of 
subjective assessments of "experts,** of project staffs (who may both design 
and implement the model during its developmental tryout), and of participants* 
instructors or supervisors. Although soft, such datfi served the formative 
evaluation needs of these model and program development projects very well. 
Moreover, as these three-year grant activities presently operate, it may not 
be feasible to expect grantees to obtain data that would rigorously support 
this federal program objective. (See last conclusion below.) 

More than 75% of the study sample (representing all eleven competition 
areas) reported outcomes that constituted a wide variety of system improvements 
which would support the third program objective, "to expand the capacity of 
the system for personnel development." However, these data will probably not 
be reported in a form that makes it feasible for federal program staff to 
extract and aggregate. 

The above types of information address information needs of the Personnel 
Preparation Program, OSBRS, and others interviewed in the course of the goal 
evaluation, if aggregated and summarized for competition areas, and for major 
types of grant activity. 

ProKram Logic and Assumptions Are Valid 

In the type of analysis characteristic of a goal evaluation, judgments of 
the validity of program logic and assumptions, and of the plausibility of 
program objectives, are based on evidence of "congruence," rather than by 
testing cause-effect linkages. In this case, the evidence presented in 
Sections III, IV, and V shows that (1) there is very good consistency bett^en 
expec; 1 and reported emphases on federal strategies through major kinds of 
grant ac ivity, and (2) the results and corroborating data that grantees* 
claimed to have will support Personnel Preparation Program objectives. In 
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short, no major incongruities with the logic model or the assumptions are 
apparent. 

One aspect of these findings requires clarification* Figures and text in 
earlier chapters showed that the most mileage toward objectives was expected 
to come from certain clusters of projects, but explained that this gross 
classification was not meant to suggest an exclusive emphasis on a particular 
strategy or objective. In fact, the Personnel Preparation Program expects 
that individual projects may implement strategies and pursue program objec- 
tives in addition to those expected to be their primary emphasis, 

This was the case for the 56 projects in the study sample v which suggests 
that support for Personnel Preparation Program objectives is broadly based. 
Such broad support also suggests that the processes and outcomes represented 
in the logic model, and elaborated in the assumptions, can occur at a level of 
performance that jus' \ /ies expectations about the Personnel Preparation 
Program* s ability to achieve its objectives. 

Model Development* Kvaluatlon. and Dissemination 
Presently Can Be Implemented Only To A Limited Extent 

The reality is that this strategy can be implemented only to a limited 
extent within a three-year grant, with most of the effort going to model 
development, promoting awareness of the model, and distributing information 
about it to potential users during its developmental tryout. Although a wide 
variety of potential users expressed serious interest in tfc<* model approaches 
that were developed by grantees in the study sample, there was little oppor- 
tunity wUhin the grant period to provide them with implementation assistance. 
Formal testing of models (and new programs) by projects in the study sample 
was not the norm, nor was objectively assessing Whether these improved 
approaches led io improved practice. 




riecommendatlons* 



There are two sets of recommendations here. Both follow from the findings 
and conclusions of the goal evaluation. The first set suggests actions that 
the federal program managers can take now to fill information gaps and improve 
program functioning. The second set suggests candidate topics that could he 
examined further in the strategy evaluation phase. 

1. utilise readily available data to address critical information 
needs about outcomes of the Personnel P reparation Program. 

Earlier in the goal evaluation, the Program Documentat ion Report pointed 
out that federal-level staff ~«nt quantitative data about what is being pro- 
duced through grants from the Personnel Preparation Program. The goal evalua- 
tion concluded that quantitative data relevant to most of these information 
needs is readily available. 

The most readily available quantitative data in summary form is what 
grantees must report once a year on the Annual Performance Report, nicknamed 
"Hie 98-199 form** by program staff. Data on the 98-199 form would meet 
information needs about numbers of individuals trained, receiving degrees, and 
certified through grants from the program, by category and level of training. 

In addition, nunbers of individual? recru*ts<i, numbers who then are 
employed in roles for which they were trained, are data that grantees in the 
study sample said they would include in th* r final reports. This information 
can be aggregated and summarized (for all grantees) to fill additional infor- 
mation gaps tha<: the goal evaluation noted. 

In the same fashion, information in grantees* final performance reports 
can be aggregated to indicate numbers of training models developed, for what 
content areas, for what target roles and levels, and the nature and reach of 
dissemination activities to Interest others in implementing the models. 



* Recommendations for improving "federal program Inputs by DPP" were presented 
earlier and are not repeated here. See Section III, pages 56 (grant award 
process), 61 (grant administration), 64 (DPP leadership), and 68 (coordina- 
tion and collaboration). 



Three Important information naps remain that the Personnel Preparation 
Program should fill: (1) the absence of rigorous data on the quality and 
effectiveness of model programs that grantees develop; (2) data to meet 
information needs of the "Leadership" competition area; and (3) empirical 
evidence from a user's perspective that the personnel trained and the models 
produced by grantees are high-quality and meet identified needs for such 
products. Some of these information gaps could be addressed during the 
strategy evaluation phase (see Candidate Topics for the Strategy Evaluation). 

2. Redefine the current strategy (and outcomes expected) for three- 
year model development, evaluation, and dissemination grant 
projects . 

The goal evaluation concluded that within a three-year grant period, 
models (and newly developed programs) can be subjected to expert peer review 
and developmental try out and revised accordingly, but may or may not be in a 
format suitable for field testing. Grantees can document outputs and outcomes 
indicated for model and program development grants, but within the limitations 
noted for the study sample. (See Tables 7 and 8 in Section IV.) 

Given the much more prominent emphasis on "models" in the FY87 announce- 
ment for Personnel Preparation Program grants (e.g., even for those program 
development, improvement, and support activities that historically have focused 
on personnel production), define acceptable quality-control procedures for 
developing new programs or improving existing ones, and for putting them in an 
exportable form. Then, provide this information to grantees, and support 
related technical assistance. (Parent projects may also perceive themselves 
as "model developers," and so would also benefit from this clarification and 
support through the TAPP contract.) 

Second, if the best of these products are to have broad impact, the 
Personnel Preparation Program needs to take these additional steps. 

a Support independent field-tests of models that appear to have 
broad significance for the field of personnel preparation. 

a Package validated models in exportable formats. 
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• Provide adopters with the assistance they need to implement the 
models successfully. 

e Provide leadership by developing and implementing a suitable 
system for sharing these promising models and practices. 

Information that would be needed to plan these steps could be developed 
during the next phase of the evaluation (see Candidate Topics for the Strategy 
Evaluation) . 

3. Extend grantees' efforts to promote system improvements. 

In FY86, grant announcements for only three of the competition areas 
directly encouraged grantees to promote system imp rovements (SEA Projects. 
Regular Educators, and Parent Organization Projects). Yet over three-fourths 
of the study sample (43 projects) said they had brought about system improve- 
ments (Strategy 7), and some of these improvements achieved a permanence during 
a three-year grant that contributed to institutio nalization (Strategy 8). 

Every possible piece of the larger system for personnel preparation 
seemed to be represented in the array of institutional contexts for these 
projects. In describing their efforts, this diverse assortment of grantees 
said that such improvements were among the most important accomplishments of 
their federally funded grants, even if these fundamental changes were 
spinoffs, or byproducts of their projects. 

Given grantees' broad commitment and ability to stimulate system change, 
the Personnel Preparation Program should consider additional ways of encourag- 
ing and assisting them to do so. For example, require the grant applicant to 
indicate how proposed activities address each of several fe deral strategies in 
a way that engages and serves different pieces of the system for personnel 
preparation: at the grantee's institution, and in the state and region that 
the institution serves. 

A second recommendation is to take steps to define and pursue a systemic 
approach to managing all aspects of the Personnel Preparation Program. This 
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means considering the system when selecting priorities for competition, 
focusing competitions, defining technical review and evaluation processes for 
gr uit applications, and so forth. 

4. Reexamine resource allocations for parent organisation projects . 

Funds allocated for the Parent competition are presently used to 
establish some grant activity in every state. In making this decision, 
federal program managers weighed the expected benefits of "getting something 
going in each state** with the obvious negatives: low visibility; and the 
impossibility of identifying and reaching a large number of parents, or of 
addressing comprehensively their needs for information and technical assis- 
tance. 

The study team's recommendation is to obtain information from the techni- 
cal assistance contractor (TAPP) on difficulties grantees. experience in trying 
to achieve project objectives given present resource allocations. Then, if 
the decision is to continue with the present allocation scheme, address the 
difficulties through the TAPP contract or sane other mechanism. 

Candidate Topics for the Strategy Evaluation 

A strategy evaluation is the second stage of the two-stage evaluation 
approach being used to review the discretionary programs under the Education 
of the Handicapped Act. The primary objective of the strategy evaluation will 
be to develop focused information on a particular program strategy that will 
respond to the needs of OS BP management and staff, and that will identify ways 
of improving the strategy. 

The mechanism is a flexible one, and may include: exploration of issues 
identified during the goal evaluation stage; exploration of alternative 
strategies to those currently employed for the program; and development of 
additional information to fill information gaps identified during the goal 
evaluation. The findings from the goal evaluation suggest that any one of the 
following topics would be good candidates for the Personnel Preparation 
Program strategy evaluation phase. 



ERIC 



-106- 

131 



Target inn critical QOjjds (strategy 2) . The nw (1986) amendments add 
language on "projected" needs to existing language on directing Personnel 
Preparation Program resources to meet shortages of personnel (all levels). 
OKB continues to push the program for information to demonstrate that its 
training grants are producing personnel to meet shortages. OSBRS/OSBP want to 
see comprehensive planning for personnel development (and its products , 
states* CSPO plans) achieve full potential so that states can be expected to 
focus efforts on meeting their acknowledged needs and priorities for personnel 
development. 

Thus, pressures are increasing for visibly and proactively managing the 
program to meet shortages. Based on information produced by the goal 
evaluation, requiring grantees to document shortages in their proposals as 
justification for funding their activities may be viewed as inadequate and 
unresponsive. 

Developing an Independent source of reliable information on critical 
shortages (current and projected) is undoubtedly beyond any single. OSBP 
division's capacity or responsibility, nevertheless, a strategy evaluation 
could develop information to improve the federal program* s understanding of 
and ability to use data that can be collected to identify personnel shortages 
in special education. 

Recruitment and retention (Strategy 1) . At present, the Personnel 
Preparation Program supports grantees* recruitment efforts by providing funds 
for stipends. Grantees acknowledge that without stipends, they would not be 
able to attract sufficient quantities of strong candidates for preparation as 
special educators, related services, and leadership personnel. However 
necessary, stipends are not a sufficient condition for effective recruitment, 
and grantees in the study sample used a variety of non-monetary tactics to 
attract, screen, and retain well-qualified and committed individuals to their 
personnel preparation programs. 

How can lessons learned from these (and other) recruitment efforts be 
identified and communicated to grantees? A strategy evaluation could Identify 
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effective recruitment efforts in areas of nationally recognized critical 
shortages, and for traditionally underrep resented groups . 

What are implications of the national clearinghouse on recruitment and 
employment opportunities (1986 amendments) for the Personnel Preparation 
Program, its strategies, and its grant activities?* k strategy evaluation 
could look at project-level and system-level elements of this strategy. 

Leadership development (Strategy 4) . Mo existing data system (including 
CSPD) provides useful information for demonstrating that leadership develop- 
ment is directed at producing doctoral-level personnel who will meet critical 
needs In the field of personnel preparation, special education research and 
program development, and so forth. To be useful for long-range planning, 
these data should enable federal program managers t estimate current needs as 
well as needs In five years, ten years, etc. 

Ho re generally, the Chief of DPP's Leadership Branch indicated the need 
for better data on what kinds of activities graduates actually carry out after 
completing doctoral programs. Do they themselves go into teaching and into 
the preparation of new personnel? Do they go into practice instead? Do they 
actually conduct leadership activities directed at improving the quality of 
programs and practices in the field? 

The study team recommends that filling these Information gaps be 
considered when making choices for focusing strategy evaluation resources. 

Model development, evaluation, and dissemination (Strategy 3) . One of 
the recommendations at the beginning of this section was that the federal 
program develop and implement a suitable system for sharing high-quality 
personnel preparation models, materials, and other products that are developed 
by its grantees. 



* The new amendments (Section 310[cD define clearinghouse activities that are 
pertinent to Strategies 1, 2, 5, and 7. 
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If this recommendation is to be implemented, information could be 
developed through the strategy evaluation that would identify important 
elements of a sound diffusion plan. For example, the strategy evaluation 
could answer these questions: 

# What federal strategies might be implemented to select the 
best of these models and programs for formal evaluation? 
For broad dissemination? 

a What lessons can be learned from other federally-sponsored 
efforts to identify, validate, and disseminate exemplary 
practice? From the considerable federal experience in 
funding and managing national demonstrations? 

The next phase of the evaluation will examine one or more strategies 
which seem to be most critical for achieving Personnel Preparation Program 
objectives. 

Of course, a strategy's "criticalness" is a matter of judgment, and must 
take into account factors beyond those addressed by a goal evaluation. These 
additional factors include changes in policies and plans that have shaped the 
Personnel Preparation Program's strategic emphases in FY87, modifications in 
individual competition areas for FY87, changes in personnel at senior 
management levels in OSBP and DPP, and so forth. 

The work group will consider the above factors and the findings, conclu- 
sions, and recommendations in this report in exploring how the strategy evalu- 
ation phase can be focused most profitably for program improvement. 
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PROTOCOL FOR PROJECT REVIEWS 
FOR THE GOAL EVALUATION OF THE PERSONNEL PREPARATION PROGRAM 

Reviewers will follow this protocol, u*ing the instrument in 
Appendix B, as closely as possible in reviewing projects. Each 
project review will have three steps, described in detail below: 

1- Determine sources of evidence and schedule data 
collection. 

2. Collect data. 

3. Complete the project review instrument. 



Determine Sources of Evidence and Schedule Data Collection 

Every review will be based on several sources of evidence. 
Where two or more sources of evidence cover the same topics, the more 
recent sources will be given credence over older ones. Sources will 
include: 



e the initial program solicitation leading to the grant 
award (e.g., FY 1984 grants announcement)? 

e initial and continuation proposals (e.g., FYs 1984, 
1985, and 1986); 

e technical review/evaluation and award documentation; 

• progress reports and performance reviews, if 
available; 

e documentation of results of grant projects, e.g., data 
on the previous year's accomplishments which are 
appended to the beginning of a continuation proposal, 
or which may be described in it; 

e products or deliverables from the grant project; 

e 60-minute telephone interview with the project 
director (or principal investogator) ; 

e 30-minute telephone interview with the DPP project 
officer or other DPP staff member, whoever is most 
knowledgeable about the project and the federal 
"program inputs" that are related to the project and 
its competition area; and 

• literature and other selected sources that are 
relevant to the project or its competition area, to 
its institutional or state context, or to presenting 
findings of the goal evaluation for clusters of 
federal strategies and grantee activities. 
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The DPP project officer(s) will be alerted that the review is to 
take place and asXed to provide contact information and to assemble 
all relevant materials. The project officer and the reviewer will 
also agree on a time for the DPP interview. 

The letter at the end of this protocol will be mai i* d 
director of each project to be reviewed, followed by a telephone call 
to (a) confirm agreement to participate, (b) re-emphasize 
confidentiality of information, and (c^ schedule a telephone 
interview. 



Collect Data 

Reviewers will study files for their assigned projects before 
interviewing project directors or DPP project officers. Then they 
will partially complete the instrument in Appendix B for eacn 
project, based on these file reviews. 

Interviews with project directors and DPP project officers have 
several purposes: 

o to confirm selected information from file reviews* 

o to explore in some depth the nature of grantee 

activity and the implementation of federal strategies 
through that activity? 

o to identify major accomplishments and longer-term 
outcomes of grantee activity? and 

o to obtain the project director's and the DPP project 
officer's perceptions of the (a) logic by which 
grantee activity and federal strategies are intended 
to further the objectives and ultimate goal of the 
Personnel Preparation Program, and (b) opportunities 
that exist for improving program plausibility and 
performance. 

The instrument in Appendix B establishes the approximate 
boundaries for the reviewer's inquiry, but does not propose a 
standard set of questions, instead, the probes and the examples or 
evidentiary information suggested in the instrument are intended to 
stimulate the reviewer to explore certain topics in depth, and to 
ensure that the reviewer's notes provide a rich source of information 
on which to base answers to questions that the goal evaluation must 
address . 

Because the instrument is designed for the reviewer (as 
respondent), there are no separate instruments for interviews of the 
project director and the DPP project officer. 
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Complete the Project Review Instrument 



The instrument contains a set of questions that will be answered 
ie reviewer, based on the evidence available. Care will be taken 
isure that the reviewer's responses: 

o clearly distinguish between facts and inferences 
from, or anaysis of, these facts? 

o use footnotes, with the appropriate citation to a 
specific document or interview to indicate the 
sources of the information; and 

o can be understood by a "cold" reader or analyst. 



Each project review will require approximately nine (9) hours, 
on the average. Reviewers will follow these guidelinas in managing 
thexr effort to complete the steps just described: 



Level of Effort 



1 



Determine sources of evidence and schedule 
data collection 



.75 hr. 



Collect data: 



- Review files and other documents 

- Interview project director. . . 

- Interview DPP project officer . 



2.75 hrs. 
1.00 hr. 
.50 hr. 



Complete the project review instrument. 



4.00 hrs. 



Average time required for one project review 



9.00 hrs. 
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SAMPLE F oRM LETTER TO PPCJECT DIRECTORS 



(Project Director's Name and Address) 



Dear 



The Office of Special Education Programs (OSEP) in the 
us Department of Education has requested reviews of 
five under the Education of the Handicapped Act. 

The purpose i- to gather information on the overall 
functioning or these programs that will help federal 
managers improve them. 

The American Institutes for Research (AIR) is P^tiete- 
ting with COSMOS Corporation, a social science 
firm, in the program reviews which began in October 1985 
and will continue for the next two years. 

The Personnel Preparation Program is the third program to 
be studied. As part of our information gathering, the 
AIR team is interviewing a sample of the .V^vnmm 
project directors by telephone about their experiences as 
funded projects. 

We would like to interview you (or your representative) 
by telephone, and hope you will agree to assist us. This 
interview will last from 45 minutes to one hour, and will 
cover the following general areas: 

e goals and objectives; 

• activities and strategies; 

e logic or rationale whereby project activities and 
strategies are to attain goals and objectives; 

e accomplishments and nature of supporting 
evidence; and 

• obstacles or frustrations encountered. 

The information you provide will NOT be used to •*»*^ 1 £ ur 
individual project, but will be aggregated with information 
from approximately 60 other projects to develop a more 
complete description of the overall functioning 
of the Personnel Preparation Program. 

We do not expect you to prepare in advance for our 
telephone call, other than identifying the person we 
should interview, if that is not yourself. However, if 
you do have on hand summary information about what your 
project has accomplished that was not included in your 
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continuation proposal— quantitative or descriptive 
summaries — we would be interested in discussing these 
with you when we telephone you. 

We will appreciate your assistance and cooperation very 
much. An AIR staff person will contact you soon to 
confirm your willingness to participate and to arrange a 
convenient time for the interview. Before interviewing 
you, that person will review information about your 
project that is already available from files at the 
federal program office. 

The enclosed brochure provides more information about the 
review approach. 

Sincerely, 



Peggie L. Campeau 



Enclosure 
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PERSONNEL PREPARATION PROGRAM GOAL EVALUATION 



PROJECT REVIEW INSTRUMENT 

The AIR reviewer will provide the responses to items in this 
instrument, first, by studying project records that DPP provides? and 
second, by interviewing DPP project officers or staff, grant project 
directors, and TAPP contract and subcontract managers. 

The instrument consists of six sections. Titles for sections B 
through F are similar to headings that appear in the "Personnel 
Preparation Program Logic Model'' and in the charts for competition 
areas in the Program Documentation Report, Sections II and III, 
respectively. 

A. Basic Descriptive Information 



About the Project Page 2 

B. Grantee Activities Page 4 

C. Intermediate Outcomes Page 11 

D. Ultimate Program Goal Page 14 

E. Federal Strategies Page 15 

F. Federal Program Inputs Page 24 



Each section (except Section A) begins with an overall protocol 
question and describes the purpose of the section. The section then 
has a list of probes that will guide the reviewer in addressing the 
topic of the section. Next, a list of illustrative evidence will help 
the reviewer to determine whether or not specific types of information 
are available on the project. 

The purpose of this project review is to obtain sufficient information 
about a project to determine its alignment with the federal program 
goal and objectives. The purpose is NOT to collect voluminous amounts 
of information for preparing detailed project descriptions. 



AIR Reviewer: 

A. BASIC DESCRIPTIVE INFORMATION ABOUT THE PROJECT 



Project Titles 

PR #: 

GOO #: 



Competition Areas 

Special Educators 

Related Services Personnel 

Rural 

Infant 

Minority 

Transition 



Leadership 

Special Projects 

Parent Org. Projects 

SEA Projects 

Regular Educators 



Agency Types 



Institution of higher education 
Private, nonprofit organization 
SEA 
Other 



Name of Agency: 

Name of Project Directors ( Dr. Ms. Mr. Mrs.) Phone 



Title s 
Address s 



Name of DPP Project Officers 



Phone s 



Name of DPP Competition Managers 



Phone s 



Other Contact(s)s 



Phone( s) s 



Fundings 

Initial Year of SEP (or BEH) Funding for This Projects 
FY 84 

Other (Explain) 

SEP Funding Amount (entire performance period): $ 
Other Funding for This Project (list below): 

Source Performance Period Amount 



Staffs 

Number FTE Type (Position & Discipline) 



Emphasis of Grantee Activity 

(New) Program Development 

Program Improvement 

Program Support 

Stipends 



Model Development and 
Dissemination 

Technical Assistance 
and Information 



Federal Strategies Receiving Most Emphasis Through Grantee Activity 

Recruitment/Retention 

Targeting Critical Needs Areas 

Model Program Development and Dissemination 

Leadership Development 

State & Professional Standards 

Parent Organization Projects 

Building System Capacity 

Institutionalization 
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B. GRANTEE ACTIVITIES 



PROTOCOL QUESTIONS W HAT MAJOR GRANTEE ACTIVITY DOES THIS PROJECT 
gEggE5ggT7 

The grantee activities characterised below are the major avenues 
through which the Personnel Preparation Program pursues federal 
strategies for achieving program objectives. These three categories 
of grantee activity appeared as column headings in Figure 1 of the 
Data Collection Plant 

1. Training program (a) development, (b) improvement, or 
(c) support, including (d) stipends 

2. Model program development, evaluation, dissemination 

3. Technical assistance and information 

The probes and the illustrative examples of evidence below will focus 
document reviews and telephone interviews on elements one would look 
for to determine if a particular activity is actually present in a 
grant (or contract). 

The examples of evidence include the "outputs" that were listed in 
charts for the competition areas in Section III of the Program 
Documentation Report. These are relatively short-term results, such as 
numbers and types of persons trained, models produced, assistance and 
information provided. Longer-term results, such as what persons DO 
afterwards who are exposed to training, models, assistance, and 
information, are addressed in a separate section of this instrument 
(see section C, "Intermediate Outcomes"). 

While the expectation is that a project will emphasize one of the 
three activities, conceivably all three activities could be present. 
Therefore, every project review will inquire about all three types of 
activities • 

NOTE* Every competition area EXCEPT "Special Projects" 
and "Parent Organization Projects" ia expected to 
fit best under 1(a), OR 1(b), OR 1(c), AND may 
also fit under 1(d). 



1(a). Is it primarily a training PROGRAM DEVELOPMENT project? 
Probes t 

- grantee's perception of thes 

— federal objectives for this type of grant 

— congruence between federal objectives and the 
project's objectives 
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- specific needs that were not met by training activity 
that existed prior to FY 1984 

- features that distinguish this "new" training activity 
from what existed prior to FY 1984 

- nature of the development process (who, what, when, 
why, how) 

- grantee's view of the most significant accomplishment or 
activity of the project 

in order tos 

"produce more high-quality personnel to meet 
current and emerging needs of handicapped 
children and youth." 

Illustrative evidences 

- number and nature of new course* and materials 
developed 

- number, types, and levels of individuals whos 

— receive training 

— complice training (or fail to) 

- types and amounts of training provided 



1(b). OR, 
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Is it primarily a training PROGRAM IMPROVEMENT proj ect? 
Probes t " 

- grantee 1 s perception of the 

— federal objectives for this type of grant 

— congruence between federal objectives and the 
project's objectives 

- specific needs that were not met by training activity that 
existed prior to FY 1984 

- features that distinguish this "improved" training 
activity from what existed prior to FY 1984 

- nature of the improvement process (who, what, when, 
why, how) 

- grantee's view of the most significant accomplishment or 
activity of the project 
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in order tot 



"produce more high-quality personnel to meet 
current and emerging needs of handicapped 
children and youth. " 

Illustrative evidences 

- number and nature of courses and materials improved 

- specific improvements in courses and materials 

- number, types, and levels of individuals who: 

— receive training 

— complete training (or fail to) 

_ types and amounts of training provided 



^- L — f| jt primarily a training PROGRAM SUP PORT project? 
probes s 

- grantee's perception of the 

— federal objectives for this type of grant 

— congruence between federal objectives and the 
project's objectives 

_ grantee's view of the most significant accomplishment or 
activity of the project 

- number of continuous years of Personnel Preparation 
Program funding for training activity 

- budgetary or contextual factors that suggest why federal 
support has been vital to the survival of the training 
program 

in order to« 

"produce more high-quality personnel to meet 
current and emerging needs of handicapped 
children and youth." 
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Illustrative evidence! 

- proportion of grantee's institutional budget that has come 
from the Personnel Preparation Program annually for the 
past "x" years 

- number of PTE special education program staff paid for 
by grant as a percentage of total staff in that program 

- numbers, types, levels of individuals who: 

— receive training 

— complete training (or fail to) 

- types and amounts of training provided 

- numbers and types of materials and courses developed or 
revised 



1(d) « Does the project emphasize the provision of STIPENDS? 
Probes i 

- dollar amount and number of stipends 

- criteria for eligibility for stipends 
in order to: 

"attract strong candidates who will prepare for, 
enter, and remain in special education to 
provide services to handicapped children and 
youth, to train others, or to lead efforts to 
expand or improve the system for personnel 
development. " 

Illustrative evidence: 

- numbers and types of individuals receiving stipends 

- ratio of stipend dollars to faculty salary/administrative 
dollars that the grant provides 

- number of participants receiving stipends as a percentage 
of total enrollment in the special education program 

- degree/certificate completion rate for students receiving 
stipends compared to rates for "regular" students in the 
special education program 

- indicators of the quality of full-time graduate students 
and PhDs that stipends attract and support 
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2. Is it primarily a MODEL DEVELOPMENT, EVALUATION, AND DISSEMINATION 
project? 

NOTE: "Special Projects" are expected to fit best in 
this category of grantee activity. However, 
projects in other categories may include model 
development, evaluation, and dissemination. 



Probes s 

- grantee's perception of the 

— federal objectives for this type of grant 

— congruence between federal objectives and the 
project's objectives 

- name and nature of model 

- aims of model stated as objectives, competencies, 
standards 

- sources utilized to establish: 

— need for model 

— "quality" of model (e.g., "Baseline Book") 

- respects in which this model has "broad significance" for 
the field of preservice and/or inservice personnel 
preparation 

_ how its effectiveness, replicability, and transportability 
are to be (or have been) established 

- nature and extent of these project activities: 

— product development (the implementation model) 

— product demonstration 

— data collection and analysis activities 

— promoting awareness of the model 

— product dissemination 

— orientation to the model for its potential consumers 

— training trainers to use the model 

— other training by grantee, using the model 

— consultation 

in order to: 

"stimulate the field of personnel preparation to 
implement such exemplary approaches, which in 
turn will make available more high-quality 
personnel to deliver services to handicapped 
children and youth." 

151 



Illustrative evidence: 



- model or product developed, i.e., nature of curricula, 
books, teacher guides, and nodules produced 

- evaluation of model or product, i.e., competencies 
demonstrated, standards met by trainees; types and levels 
of personnel for which model is applicable; indicators of 
"quality" met by product 

- dissemination of model or product, i.e., number and types 
of materials, presentations, or publications disseminated 
over an "X"-year period to promote replication of the 
model practices; number and types of dissemination 
activities planned; extent of ongoing capability to 
disseminate model 

- dissemination targets, i.e., number and type of 
dissemination efforts directed at special educators, 
related services personnel, administrators, trainers, 
developers, parents, others; number, nature, and 
geographic distribution of dissemination activities 

- replication, i.e., extent to which concepts and procedures 
contained in the model are actually used (conceptually or 
instrumentally) to improve subsequent training projects 



3. Is it prima ri ly a TECHNICAL ASSISTANCE AND INFORMATION project? 

NOTE: parent Organization Projects" and TAPP are 
expected to fit best in this category of 
grantee activity. However, projects in other 
categories may include technical assistance and 
information services • 

Probes : 

- key features of technical assistance content and approach 

- nature and extent of these project activities: 

— assistance or informational activities aimed at 
specific sites, audiences 

— idc -ification of effective strategies for providing 
assistance and information aimed at specific sites, 
audiences 

- characteristics of target audience(s) for training and 
information 
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in order tot 

"stimulate improvements in training and system 
capacity that make available more personnel and 
parents who are able to provide effective 
education and related services to handicapped 
children and to interact effectively with the 
system on their behalf." 

Illustrative evidence: 

_ numbers, types, levels of individuals who, over an M X U 
year period* 

— request assistance and information 

— receive assistance and information 

- number and nature of assistance and information provided 
over an "X'-year period 

- numbers and types of materials, curricula, programs, 
presentations, or publications developed or revised over 
an "X"-year period 

- competencies and knowledge demonstrated by those assisted 

_ examples, if any, of perceived usefulness of assistance 
and information 
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C. INTERMEDIATE OUTCOMES 



PROTOCOL QUESTION; TO WHAT EXTENT DID THE PROJECT ACHIEVE RELEVANT 
' INTERMEDIATE OUTCOMES? "" ' 

(If the judgment is premature, describe when the 
outcomes will occur and why the judgment is 
premature. ) 

These are the longer-term results that may be attributable to the 
project (at least partially), and that fall on a continuum somewhere 
between short-term project results ("outputs-) and the ultimate goal 
of the Personnel Preparation Program. Longer-term results include 
effects that appeared in the "outcomes" column of the charts for the 
competitions in Section III of the Program Documentation Report. 

These effects lie beyond merely exposing people to training, models, 
assistance, and information. Instead, "intermediate outcomes" address 
the questions, "What do persons trained do afterwards? Are model 
training programs replicated? By whom? With what effect on quality 
of personnel trained? On system capacity? For whose benefit? How 
does this fit or not fit with federal program aims and strategies?" 



1. Did it increase the QUANTITY of qualified personnel? 

DEFINITION* Producing specialized personnel who are (a) 

qualified to meet current and emerging needs of 
handicapped children and youth, and who (b) go 
on to provide them with educational and related 
services (all levels), in areas appropriate to 
their qualifications. 

Illustrative evidences 

- documentation that individuals who complete training 
programs (all levels) enter or are retained in the special 
education/related services field, e.g., numbers, types, 
and levels of individuals whoi 

— indicate their intention to enter careers in special 
education upon completion of training 

— indicate their intention to go on to more advanced 
levels of preparation for careers in special 
education 

— actually go on to serve handicapped children and youth 
in areas appropriate to their qualificatfons 

- documentation, if any, of post-graduation activities of 
recipients of "leadership" stipends 
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documentation that the types and levels of personnel 
completing the specialized training are congruent with, 

—critical needs identified by grantees 

— critical needs identified by others 

extent to which such qualified personnel report that their 
training assisted them -,n ths job 



2. Did it improve the QUALI TY of personnel? 

definition: Producing leadership, knowledge, models, and 
DEFINITION: proa^ g^ that (a) stiniu i a te other training 

programs to use state-of-the-art personnel 
preparation practice at all levels, which in 
turn (b) produce high-quality personnel who (c) 
meet or exceed professional and state standards 
for specialized personnel to serve handicapped 
children and youth. 

Illustrative evidence: 

- use by training programs of state-of the art practices 
that are appropriate for preservice and ineervice training 
of specialized personnel (all levels) 

- increased numbers of persons who, upon completion of 
training in these programs, meet or exceed professional 
and state standards and other indicators of "quality for 
various types of specialized personnel (all levels) 

- increase in doctoral level personnel qualified to train 
teacher trainers, do research, and administer special 
education programs 

3. Did it EXPAND SYSTEM CAPACITY for personne l preparation? 

DEFINITION : Influencing the system directly or indirectly 
to (a) identify and meet local, state, 
regional, and national needs for trained and 
certified personnel, and for regular 
educators qualified to work with handicapped 
children and youth in least restrictive 
environments, and to (b) sustain these 
personnel preparation programs after federal 
support for them ends • 

illustrative ev-dence: 

- sources, amounts, and percentages of grantee institution's 
total special education program financial support from 
nonfederal sources for the past M X" years 

1 ^ 
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extent to which "graduates" of parent training projects engage 
in productive interactions with the educational system on 
behalf of their handicapped children 

institutional policies and actions taken as a result of 
grantee activity that increase the system's responsibility 
and capacitv ?or personnel preparation, e.g.,i 

— support for personnel development (all levels) 

— financial assistance for training program participants 

— support for model development, evaluation, 
demonstration, and dissemination 

— support for technical assistance and information 
services related to personnel development (all levels), 
including regular educators and parents 



156 



D . ULTIMATE PROGRAM GOAL 




1. 



Was it the effective use of skills acquired in the training 
program? 

Probes : 

- direct evidence regarding handicapped students' 

educational attainments 

— economic self-sufficiency 

— social self-sufficiency 

that can be linked causally toj 

- the use of skills acquired in training. 
Illustrative evidence: 

_ ©valuation studies at the school, classroom, and 
community level 
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E. FEDERAL STRATEGIES 
PROTOCOL QUESTION* WHAT FEDERAL STRATEGIES WERE USED BY THE PROJECT? 



ALSO, what is the logic or rationale whereby 
the project's activities (and the federal 
strategies they represent) are intended to 
further one or more of the three objectives 
of the personnel Preparation Program? 

By this point in the instrument, the reviewer will haves 

• established the type of grantee activity represented by 
the project; 

• collected information regarding intermediate outcomes, 
which are the longer-term results that further federal 
program objectives (increase the QUANTITY and improve the 
QUALITY of personnel trained, and expand SYSTEM CAPACITY 
for personnel development), and 

• identified outcomes (if any) that further the ultimate 
goal of the Personnel Preparation Program — "to enhance 
education and related services for handicapped children 
and youth through the preparation of specialized 
personnel . " 

The purpose of this section of the instrument is to guide the reviewer 
in (1) establishing the federal strategies represented in the grantee 
activity, as the project has actually operated, and (2) making 
explicit the logic (rationale) whereby grantee activity is intended to 
further the objectives and ultimate goal of the Personnel Preparation 
Program . 

The Program Documentation Report lists eight (8) strategies that the 
Personnel Preparation program pursues to achieve the three program 
objectives. Figure 1 in the Data Collection Plan shows the presumed 
relationships among the three objectives, the eight strategies, and 
major categories of grantee activity. It implies that all projects in 
a particular cluster will represent best the strategies associated 
with that cluster. For example, "SEA Projects" and "Regular Educator" 
projects will best exemplify the three system capacity strategies, 
presuming these grant activities are implemented as intended. 

In actual practice, however, the relationships will not be as clearcut 
as Figure 1 suggests. The eight strategies, when implemented through 
grantee activities, may turn out not to be mutually exclusive. 
Moreover, individual projects in the clusters depicted in Figure 1 may 
emphasize strategies that are different from, or in addition to, the 
strategy(ies) attached to their particular priority area. Therefore, 
the reviewer will attend to ALL EIGHT strategies in reviewing grantee 
records and interviewing the project director, regardless of where the 
project was originally presumed to fit in Figure 1. 
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probes i 

- methods implemented to* 

~ attract and retain strong candidates until they 

complete training (all levels) 
- encourage "completers" to enter roles appropriate to 

their training 

- criteria used to identify strong candidates 

so that: 

-strong candidates are more likely to prepare 
for, enter, and remain in careers in special 
education, and thereby increase the (QUANTITY) 
of individuals specially trained to serve 
handicapped children and youth. 

Evidence : 

- actions taken by grantee to attract and retain 
strong candidates 

- ho* these actions were supported 

- numbers, types, and levels of individuals recruited by 
grantee 

- documentation, if any, that these actions increased 
training enrollment and completion 

- documentation, if any, that "completers" i 

— indicate their intent to fill roles appropriate to 
their training and qualifications 

actually take on such roles 

- document the logic/rationale whereby grantee activity 
(including the above elements) furthers one or more 
of the three federal program goals 

Plausibility and performance issues » 

- gaps or flaws in the above logic/ rationale 

- major constraints or frustrations experienced, addressed, 
anticipated in attempting to recruit and retain strong 
candidates 
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2. Was there evidence that TARGETING CRITICAL NEEDS was utilized? 



Probes s 

- extent to which grantee implemented systematic procedures 
for assessing local, state, regional personnel training 
needs (or needs for models, or for technical assistance 
and information) 

- extent to which systematically assessed needs affected t 

— training design 

— training opportunities provided in the locale, state, 
or region 

so that: 

"personnel preparation (or model development, or 
technical assistance and information) in areas 
of critical need will (increase the QUANTITY) of 
these types of qualified personnel (or models, 
or technical assistance and information).* 1 



Evidence s 

- actions taken and sources utilized to assess specific 
needs for personnel (or models, technical assistance, 
information) 

- how project activities are presmued to ameliorate 
these needs 

- document the logic/rationale whereby grantee activity, 
including the above elements, furthers one or more of the 
three federal program objectives 

Plausibility and performance issues: 

- gaps or flaws in the above logic/ rationale 

- major constraints or frustrations experienced, addressed, 
anticipated in attempting to target project activities to 
address areas of critical need 



3. Was there evidence of MODEL DEVELOPMENT, EVALUATION, 



in order tot 
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"promote the refinement and distribution of 
improved training methods of broad significance 
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for the field of preservice and inservice 
personnel preparation (all levels), which will 
(a) encourage replication of best practices by 
other training programs, leading to (b) 
inmproved QUALITY of personnel trained in these 
programs . N 

Reference! Information collected for Section B, "Grantee 
Activities,** Question 2. 

Evidence : 

- description of model 

- how model development, evaluation, demonstration, 
dissemination were supported 

- ways in which the model has "broad significance'* 

- documentation, if any, that the model is effective, 
replicable, transportable 

- documentation, if any, of target market interest in 
and/or actual implementation of model 

- document the logic/rationale whereby orantee activity 
(including the above elements) furthers one or more of 
the three federal program objectives 

Plausibility and performance issues: 

- gaps or flaws in above logic/rationale 

- major constraints or frustrations experienced, addressed, 
anticipated in attempting tos 

— develop model 

— determine its effectiveness and replicability 

— sustain demonstration and dissemination activities 



4. Was there evidence of LEADERSHIP DEVELOPMENT? 
Probes 

- methods used to develop doctoral and postdoctoral 
personnel who will go on to: 

— train teachers 

— do research 

— administer programs 

in order to: 



lfil 



"promote use of state-of-the-art methods in 
personnel preparation (all nevels), leading to 
improved QUALITY of these personnel." 



Evidence s 

- description of the leadership development activity 

- how the activity prepares students to promote 
state-of-the-art practice, e.g., does it train doctoral 
and postdoctoral students tos 

— do research? 

— identify best practices? 

— utilize effective training and administrate "e methods? 

— other? 

- documentation of the logic/rationale whereby grantee 
activity (including the above elements) furthers one or 
more of the three federal program objectives 

Plausibility and performance issues t 

- gaps or flaws in above logic/rationale 

- time frame for progressing to positions of leadership in 
the field of personnel preparation 

- major constraints experienced, addressed, anticipated in 
attempting to provide doctoral and postdoctoral programs 
that » 

— incorporate and promote state-of-the-art practice 

— prepare leadership personnel for careers in research, 
administration, and personnel preparation 



5. Was there evidence that STATE and PROFE SSIONAL STANDARDS were 
utilized? 

in order tos 

". . . ensure the QUALITY of personnel trained." 
Evidences 

- presence of assurances in proposal (s) 

- source(s) of standards and other quality indicators, e.g., 
"Baseline Book," latest guidelines from professional 
organisations , etc . 
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- nature and extent of interaction with standards- eetting 
agencies and organizations 

- documentation of how standards and/or indicators of 
quality were utilized and with what results 

- document the logic/rationale whereby grantee activity 
(including the above elements) furthers one or more of the 
three federal program objectives 

Plausibility and performance issues: 

- gaps or flaws in above logic/ rationale 

- major constraints experienced, addressed, anticipated in 
utilizing state and professional standards, or other 
indicators of quality 



6. Was there evidence of PARENT training and information activity? 
in order to: 

"help (parents) influence the SYSTEM to develop 
and exercise its CAPACITY to meet the needs of 
their handicapped children." 

Reference: Information collected for Dection B, "Grantee 
Activities," Question 3. 

Evidence : 

- description of the parent training, assistance, and 
information activity 

- specific competencies and knowledge that parents acquired 
as a result of participating in the project 

- indications of how, over "X" time, the training, 
assistance, and information provided is expected to help 
them influence the educational system to meet the needs of 
their handicapped children 

- indications of the extent to which the project serves 
parents of all handicapped children in all categories 

- document the logic/ rationale whereby grantee activity 
(including the above elements) furthers one or more of the 
three federal program objectives 

plausibility and performance issues: 

- gaps in the above logic/rationale 
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- major constraints experienced, addressed, anticipated in 
attempting to design and implement training and 
information programs that equip parents of handicapped 
children with* 

— knowledge of their children's ri *hts and the system's 
responsibilities 



— effective strategies for working with the system on 
behalf of their children 



7- Was their evidence of CAPACITY BUILDING? 

in order tot 

"increase the system's capacity to meet local, 
state, and regional needs for trained and 
certified personnel, and for regular educators 
qualified to work with handicapped children and 
youth in least restrictive environments (which 
in turn) will INCREASE SYSTEM CAPACITY for 
personnel development." 

Reference: Information collected for Section C, 
"Intermediate Outcomes, M Question 3. 

Evidence : 

- ways in which the capacity of the system for personnel 
preparation improved as a result of support by the 
Personnel Preparation Program 

- document the logic /rationale whereby these improvements 
are expected to provide more and better training 
opportunities to meet local, state, and regional needs for 
specialized personnel, including regular educators 

Plausibility and performance issues i 

- gaps or flaws in above logic/rationale 

- major constraints or frustrations experienced, addressed, 
anticipated in attempting to improve the capacity of the 
system for personnel development 



S. Was there evidence of INSTITUTIONALIZATION? 
so that: 
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the CAPACITY OP THE SYSTEM for personnel 
development can "sustain personnel preparation 
programs ... at all levels . . . after federal 
support for them ends*" 

Evidence t 

- any increase in the level of nonfederal support for 
personnel development 

- changes in organisational structures, institutional 
actions, policies, or other changes that indicate 
increased commitment to personnel development, including* 

increased number of budgeted positions for special 

education teacher trainers 

training opportunities for special education and 

related services personnel (all levels), regular 
educators, parents 

— model demonstration and dissemination 

technical assistance and information services 

- document the logic/rationale whereby these changes may 
enable the system to sustain personnel preparation (all 
levels, all areas) if federal support for it ends 

Plausibility issues s 

- gaps or flaws in above logic/rationale 

- constraints on the system, e.g., inertia of system vis a 
vis emerging or new roles for special educatgion and 
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REVIEWERS Summarize the linkages between this grantee's activities 
(representing "N" federal strategies) and the three 
federal program objectives. 

Further 

Represent- These Indicate 
ing These Program Page Ref. 
Grantee Activities Strategies Objectives in this 

(*) (**) (***) Instrument 



{*) Identify major activities and code each one as indicated below. 



1(a) NEW program development? 

1(b) program IMPROVEMENT? 

1(c) program SUPPORT? 

1(d) providing STIPENDS? 



2 model development, evalua- 
tion, or dissemination? 

3 technical assistance/info.? 



(**) Opposite each activity in the left-hand column, write the number 
of the strategy f ies) represented by that activity. 

1 recruitment/retention 

2 targeting critical needs 

3 model development, evaluation, dissemination 

4 leadership development 

5 state and professional standards 

6 parent organization projects 

7 capacity building 

8 institutionalization 

(***) In this column, enter either QUAN, QUAL, or SYS, which 
correspond to the three federal program objectives. 
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F. FEDERAL PROGRAM INPUTS (BY DPP) 
PKOTOCO, QUESTION t ^^^^^gggS^g^Ig 

PROJECT; fOMPEf IT10M 7 — 

The purpose of this section of the instrument is to collect 
information about Personnel Preparation Program activity in the 
Division of Personnel Preparation (DPP) that helps the reviewer 
analyze findings from data collected in the instrument. 

For example, DPP actions and policies may help explain linkages (or 
linkage failures) between grantee activity, federal strategies to be 
pursued through that activity, and the presence (or absence) of 
outcomes that further the three objectives of the Personnel 
Preparation Program. 

The reviewer will obtain much of the information for this section of 
the instrument from documents, such as the FY 1984 grant 
announcements, technical review and evaluation plan, prefundxng review 
packages, and other award documentation. 

In addition, reviewers will interview DPP staff and grantees for each 
project in the sample to give them an opportunity to comment on any ox 
these federal level activities that they believe have enhanced or 
hindered their ability (or the system's inclination) to further the 
three program objectives. 

REVIEWER* Focus discussions with DPP staff and grantees on 
factors that may help explain (1) congruities or 
incongruities between program intent and actual 
implementation, and (2) the ease or difficulty of 
furthering program objectives. 

1. How was the GRANT PROCESS carried out for the p roject (and the 
competition)? * 

in order toi 

further federal program objectives and to pursue 
federal strategies that are linked to this type of 
grant activity/competition. 

Potentially productive topics includes 
For the competition, 

- focus of competition, and changes in focus over time 
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- logic or rationale whereby the competition focus is 
intended to further program straegies and objectives 

- related constraints and gaps that characterize this logic 
in actual practice 

- clarity of application package 

- degree to which the "technical review and evaluation 
plan"* 

— reflected program regulations and policies 

— was implemented as intended 

Examples: selection of reviewers and formation of 
panels, their orientation and instruction; 
guidelines for project officer recommendations, 
management reviews, and preparation of prefunding 
packages 

- number of awards made 

- total amount of funds awarded to new and continuation 
grants 

- average size of grant, and range (lowest, highest award) 



For the project, 

- overall rating of proposal 

- significant aspects of proosal that were negotiated and 
changed in the final award 



2. How was GRANT ADMINISTRATION carried out for this project (and 
for this competition)? 

Potentially productive topics includes 

- quantity and content of contacts between DPP staff and 
grantee since initial award 

- perceived "quality" and utility of contacts 

- relevance of contacts for furthering federal program 
objectives 

- nature of data on project accomplishments that are* 

— requested by DPP 

— reported by grantee 

— summarized by DPP 

— actually used by DPP (HOW USED? 
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3. What other activities of the Personnel Preparation Program t 

- influenced the grant activity/ competition 

- help explain perceived gaps between expectations and 
reality for this project/competition? 
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APPENDIX C 
Competition Areas 



Table C-l. 



Summary of Funding History 
for Competition Areas 



Charts for Each Area 



Figure C-l. 
Figure C-2. 
Figure C-3. 
Figure C-4 . 
Figure C-5. 
Figure C-6 . 
Figure C-7. 
Figure C-8 . 
Figure C-9 . 
Figure C-10. 
Figure C-ll. 



Special Educators 

Related Services 

Rural 

Infant 

Transition 

Minority 

Leadership 

Special Projects 

Regular Educators 

SEA Projects 

Parent Organization Projects 
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CONPETITIDN AREAS 

SPECIAL EDUCATORS 

no* 

cent 

INFANTS 



OOAt 

MI NOR I TY 
nav 

cont 

RELATED SERVICES 

nmw 

Bout 

SEA PROJECTS 

nav 

oont 

PARENT ORG PROJECTS 

no* 

cont 

LEADERSHIP 
nmw 

cont 

SPECIAL PROJECTS 

nav 

cont 

RURAL 
new 

cont 

REGULAR EDUCATORS 

now 

cont 



Table C-l. Sunwiary of Funding History for Competition Areas 
1903 9 1983 » 1904 0 1984 • 1985 # 1985 8 1986 # 1986 8 



187 811,896,471 
8 M 



8 
8 



9 
0 



29 81,339,288 
0 88 



27 01,895,887 
8 88 



35 
8 



8 
8 



12 
0 



•2, 124,658 



47 83, 726, 967 
8 80 



32 83, 067, 816 
24 81,521,000 



8668, 486 



131 88,196,833 
185 810,962,216 



0 

0 



0 
0 



0 
0 



15 
10 



80 

80 



27 01,397,830 
26 01,262,735 



22 81,576,864 
25 81,891.499 



41 83,745, 212 
32 82, 808, 720 



40 83, 289, 216 
47 93, 762, 63 



21 61,729, 197 
45 63, 978, 320 



80 



8996, 453 



116 88,014,673 
309 619, 298, 538 



15 61,819,444 
9 80 



25 41,580,0:7 
51 82.571,541 



9 8664, 417 

47 63, 400, 653 



11 81,084,804 
64 65, 137, 685 



12 



13 



6 
0 



9 
25 



8885, 697 
86, 987, 298 



8921,726 
64, 939, 922 



9564,899 
60 



6633, 846 
81,613, 460 



155 011,064,026 
247 016, 770, 345 



15 61,082,888 
15 61,216,416 



19 
0 



18 
11 



a 

6 



7 
24 



81,561,886 



28 62, 880, 880 

49 62, 929, 178 



39 92,191,843 



64, 340, 402 
01, 136, 630 



19 62, 250, 
52 64, 



20 61,750,000 

34 62,750,1 



6564, 624 
6463, 668 



6580,000 
01,750,800 



TRANSITION 
nmw 

cont 



0 
0 



9 
9 



8807,663 
80 



0 

9 



80 

6607, 373 



0 

9 



6031, 106 



PROGRAM ASSISTANCE 

nmw 

cont 

OTHER EXPENSE CATEGORIES 

nmw 

oont 

other 



444 023, 740, 360 



0 

185 



1 
8 



60 

09,881,357 



6129, 743 
60 
658,800 

862 655,416,682 



8 
6 



0 

1 



60 



6732, 571 
868,800 



0 

0 



0 

1 



00 



0750,660 
6325. i 



TOTALS 



637 649. 280. 047 



860 660. 945. 694 



888 661,246. 



Note: Entries in the FYS 6 column for some competitions way vary slightly from actual awards as a result of grant 

negotiations that were underway while this report was being prepared, 172 

BEST COPY AVAILABLE 



COMPETITION OBJECTIVE 



INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



To provide pr user vice 
training fpr personnel 
preparing for employment 
as special educators of 
handicapped children 
and youth 
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L egislation and Regulations 

• P.L. 91-230, Sections 631, 
632, and 634, Part D, as 
amended by P.L. 93-380 
(1974), P.L. 98-199 (1983) 

• 34 CFK Part 31 B 

• EDGAR, 14 CFR Parts 74, 75, 
77 and 78 

Funding (FY86) 

• (See Table C-l.) 

Eligibility Require me nts 

• IHEs, public and private 
colleges, SEAs, LEAs, 
other nonprofit public 
and private agencies 

Conditions That Must Be Met 

• Applications must show 
evidence (row data col- 
lected annually that 
need does exiht, and 
reflect priorities 
identified in CSPD por- 
tion of state plans 

Repo rt ing^ jtenjjlreroent s 

• Annual Financial Status 
Report 

• Final Performance Report 
(within 90 days after 
project terminal ion) 

• Annual Performance Report 
(continuation projects) 

M onitoring 

• Grants are assigned to 
project officers to 
monitor by state. 

• Project officers do tele- 
phone monitoring of 
grantees. 



App l icat io ns Fu n ded (New, FY86) 

• (See Table C-l.) 

Rev l ew C r 1 ter iji 

• Extent of need (25 »wints) 

• Program content (!!♦ points) 

• Operation plan (15 points) 
» Evaluation plan (15 points) 

• Quality of key personnel 
(15 points) 

• Adequacy of resources 
(5 points) 

9 Budget *i0 points) 

Review Process (FY86) 

• Applications are reviewed by 
external peer review panels 
consisting of 3 reviewers 
(limited to special educa- 
tion professionals). 

• Project officers' recommen- 
dations are based heavily on 
raw and standardized score 
ratings by peer review 
panels and on panels' 
narrative Justifications. 

• Review process usually takes 
3 months or more. 



• Number and category of 
students trained 

• Number and nature of mate- 
rials and programs devel- 
oped , revised, or supported 

• Number and degree level of 
students receiving stipends 



• Increase in special educa- 
tion personnel qualified 
to educate handicapped 
children and youth 
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Figure C-l. Special Educators 



COMPETITION OBJECTIVES 



INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



To provide preservice 
preparation to individ- 
uals who provide develop- 
mental, corrective, and 
other supportive services 

To improve the quality 
and increase number of 
professionals who pro- 
vide special edur.it ion 
related services 



r 
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Legislation and Regulations 

• P.i. 91-230, Sections 631, 
632, and 634, Part D, as 
amended by P.L. 91-380 
(1974), P.L. 98-199 (1983) 

• 34 CFR Part 318 

• EDGAR 34 CFR Parts 74, 75, 
77, and 78 

Funding (FY8b) 

• (See Table C-l.) 

Eligibility Requirements 

• IKEs, public and private 
colleges, SEAs, LEAs, 
other nonprofit public 
and private agencies 

Conditions That Must Be Met 



• Applications must show 
evidence from data col- 
lected annually that 
need does exist* and 
reflect priorities 
identified in CSPD 
portion of state plans. 

Reporting Requirements 

• Annual Financial Status 
Report 

• Final Performance Report 
(within 90 days after 
project termination) 

• Annual Performance Report 
(continuation projects) 

Monitoring 



Grants are assigned to 
project officers to 
monitor by state. 

Project officers do tele- 
phone monitoring of 
grantees. 



Applications Funded (New, FV86) 

• (See Table C-l*) 

Review Criteria 

• Extent of need (25 points) 

• Program content (15 points) 

• Operation plan (15 points) 

• Evaluation plan (15 points) 

• Quality of key personnel 
(15 points) 

• Adequacy of resources 
(5 points) 

a Budget (10 points) 

Review Process (FY 86) 

a Applications are reviewed by 
external peer review panels 
consisting of 3 reviewers 
(limited to related ser- 
vices professionals). 

• Project officers 1 recommen- 
dations are based heavily on 
raw and standardized score 
ratings by peer review 
panels and on panels' 
narrative justifications. 

• Review process usually takes 
3 months or more. 



• Number and category of 
students trained 

• Number and nature of 
materials and programs 
deve loped , re vi sed , or 
supported 

• Number and degree level 
of students receiving 
stipends 



• Increase in personnel 
qualified to provide 
developmental, corrective, 
and other special educa- 
tion related services to 
handicapped children and 
youth 
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Figure 02. Related Services 



COMPETITION OBJECTIVE 



INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



To provide preservice 
training for personnel 
for rural arras 
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Leg lslat io n jiml Re^ulat ions 

• P.Lp 91-230, Sections 631, 
612, and 634, Part D, as 
amended by P.L. 93-180 
(1974), P.L< 98-199 (1983) 

• 34 CFR Part 31 C 

• EOT AH 34 CFR Parts 74, 75, 
77, and 78 

Funding (FY 86) 

a (See Table C-l.) 

Eligibility Requirement a 

• IHEs, public and private 
college*, SEAs, l.EAs, 
other nonprofit public 
and private agencies 

Co nditions That M ust Be Met 

• Applications must show 
evidence from oat a col- 
lected annually that 
need does exist, and 
reflect priorities 
identified in CSPP 
portion of state plans. 

• Annual Financial Status 
Report 

• Final Performance Report 
(within 90 days after 
project termination) 

• Annual Performance Report 
(continuation projects) 

Monitoring 

m Grants are assigned to 
project officers to 
monitor by state* 

• Project officers do 
telephone monitoring 
of grantees. 



Applications Fu nded !N*w. FY86) 

• (See Table C-l,) 

Revi ew Criteria 

• Extent of need (25 points) 

• Program concent (15 points) 

• Operation plan (15 points) 

• Evaluation plan (15 points) 

• Quality of key personnel 
(15 points) 

• Adequacy of resources 
(5 points) 

• Budget (10 points) 

Rev lew Process (FY86) 

• Applications are reviewed by 
external peer review panels 
consisting of 3 reviewers. 

• Project officers 1 recommen- 
dations are based heavily on 
raw and standardised score 
ratings by peer review 
panels and on panels* 
narrative justification. 

• Review process usually takes 
3 months or more. 



• Number and category of 
students trained 

• Number and nature of mate- 
rials and programs devel- 
oped, revised, or supported 

• Number and degree ',evel of 
students receiving stiper,d» 



Increase in personnel 
qualified to fill a 
variety of rural-specific 
roles with hanoi capped 
students, parents, peers, 
and administrators 
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Figure C~3. Rural 



COMPETITION OBJECTIVES 



INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



• To provide preservice 
preparation of personnel 
to serve handicapped and 
at-risk newborn and 
infants 

• To prepare personnel to 
provide short-term or 
long-tern special educa- 
tion and related services 
(as in an intensive care 
nursery or a preschool 
program) 

• To prepare per bonne 1 ior 
employment in programs 
characterized by inter- 
action of the medical, 
educational , and related 
service, coramunl ties, and 
Involvement of parents 

or guardians 
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Legislation and Regulations 

• P.L. 91-230, Sections 631. 
632, and 634 9 Part D. as 
amended by P.L, 93-380 
(1974), P.L. 9*M99 (1983) 

• 34 CFK Part 3)8 

• EDGAR, 14 CP> Parts 74, 75, 
77, and 78 

Funding (FY 86) 

• (See Table 1>1.) 

Eligibility Requirements 

• IHEs, public and private 
colleges, SEAs, LEAs, 
other nonprofit public 
and private agencies 

Condi tions That Must Be Met 

• Applications must show 
evidence from data col* 
lected annually that 
need does exist, and 
reflect priorities 
identified in CSPI) 
portion of state plans. 

• Annual Financial Status 
Report 

• Final Performance Report 
(within 90 days after 
project termination) 

• Annual Performance Report 
(continuation projects) 

M onitoring 

• Grants are assigned to 
project officers to 
monitor by state. 

• Project officers do 
telephone monitoring 
of grantees. 



Applications Funded (New, FY86) 

• (bee Tabl. C -l.) 

Review Criteria 

• Extent of need (25 points) 

• Program content (15 points) 

• Operation plan (15 points) 

• Evaluation plan (15 points) 

• Quality of key personnel 
(15 points) 

• Adequacy of resources 
(5 points) 

• Sudget (10 points) 

Review Process (FY86) 

• Applications are reviewed by 
external peer review panels 
consisting of 3 reviewers 
experienced in the prepara- 
tion of personnel to serve 
newborn and infant handi- 
capped and at-risk. 

• Project officers* recommen- 
dations are based heavily on 
raw and standardized score 
ratings by peer review 
panels and on panels' 
narrative justifications. 

• Review process usually takes 
3 months or more. 



• Number and category of 
students trained 

• Number and nature of mate* 
rials and programs devel- 
oped, revised, or supported 

• Number and degree level of 
students receiving stipends 



Increase in personnel qual- 
f fied to provide short- and 
long-term care to handicap- 
ped and at-risk newborn and 
infants 

Increase in qualified per- 
sonnel working in progiams 
characterized by inter- 
action between medical, 
educational, and related 
services communities, 
and parental or guardian 
involvement 



Figure C-4. Iniant 



COMPETITION OBJECTIVE 



INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



To provide preservice 
training for personnel 
preparing for employment 
in programs designed to 
prepare handicapped 
youth for community 
placement and adjustment 
to the community setting. 



Legislation and Regulation s 

• P.L. 91-230, Sections 631, 
632, and 634 , Part D, as 
amended by P.L. 93-380 
(1974) f P.L. 98-199 (1983) 

• 34 CFR Part 318 

• EDGAR, 34 CFR Parts 74, 75, 
77, and 78 

Funding (FY84) 

• (See Table CM.) 

Eligibili ty R equirements 

• IHEs, public and private 
colleges, SF.As, LEA, other 
nonprofit public and 
private agencies 

Conditions That v Must Be _Met 

• Applications must show 
evidence from data col- 
lected annually that need 
does exist, and reflect 
priorities identified in 
CSPD portion of state 
plans. 

Reporting Re^uj remerUs 

• Annual Financial Status 
Report 

• Final Performance Report 
(within 90 days after 
project terminal ion) 

• Annual Performance Report 
(continuation projects) 

Monitoring 

• Grants are assigned to 
project officers to 
monitor by state. 

• Project officers do tele- 
phone monitoring of 
grantees. 



A pplications Funded (New, FY84) 

• (See Table C-l.) 

Review Criteria (FY 84) 

• Extent of need (20 points) 

• Program content (15 points) 

• Operation plan (15 points) 

• Evaluation plan (15 points) 

• Quality of key personnel 
(15 points) 

• Adequacy of resources 
(10 points) 

• Budget (10 points) 

R eview Process (FY 84) 

• Applications are reviewed by 
panels consisting of 3 review- 
ers with appropriate expertise 
(one of whom must be non- 
federal) * 

• Project officers 9 recommenda- 
tions are based heavily on 
raw and standardised score 
ratings by peer review panels 
and on panels 9 narrative 
just i f ications. 

• Review process usually takes 
3 months or more. 



Number and category of 
students trained 

Number and nature of 
materials and programs 
developed, revised, or 
supported 

Number and degree level 
of students receiving 
stipends 



• Increase in special edu- 
cation and related 
services personnel 
qualified to prepare 
handicapped youth to 
meet adult roles 
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Figure C-5. Transition 



COMPETITION OBJECTIVES 



INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



To provide preservice pre- 
paration of special educa- 
tion and/or related person- 
nel to educate minority 
children 

To provide preservice pre- 
paration to groups that 
have been under re pre sen ted, 
such as ©embers of racial 
or ethnic minorities 
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Legislation and Regulations 

• P.L. 91-230, Sections 631, 
632 » and 634, Part D, as 
amended by P.L. 93-180 
(1974), P.L. 98-199 (1983) 

• 34 CFR Part 318 

• EDGAR 34 CFR Parts 74, 75, 
77, and 78 

Funding (FY86) 

• (See Table CM.) 

£- l iability Requireme nts 

• IHEs, puMic and private 
colleges, SEAs, LHAs, 
other nonprofit public and 
private agencies 

Conditions That Must Be Met 

• Applications roust show 
evidence from data col- 
lected annually that need 
does exist, and reflect 
priorities identified in 
CSFD portion of state 
plans. 

Re po r t i n g Regu i regents 

• Annual Financial Status 
Report 

• Final Performance Report 
(within 90 days after 
project termination) 

• Annual Performance Report 
(continuation projects) 

Monitoring 

• Grants are assigned to 
project officers to 
monitor by state* 

• Project officers do tele- 
phone monitoring of 
grantees. 



Applications Funded (New, FY86) 

• (See Table C-l.) 
Review Criteria 

• Extent of need (25 points) 

• Program content (15 points) 

• Operation plan (15 points) 

• Evaluation plan (15 points) 

• Quality of key personnel 
(15 points) 

« fiequacy of resources 
( *> points) 

• Budget (10 points) 

Review Process (FY86) 

• Applications are reviewed by 
external peer review panels 
consisting of 3 reviewers 
(to include 1 reviewer 
active in the prepaiation 
of minority handicapped 
children). 

a Project officers 1 recommen- 
dations are based heavily on 
raw and standardised score 
ratings by peer review 
panels and on panels* 
narrative justifications. 

• Review process usually takes 
3 months or more. 



Number and category of 
students trained 

Number and nature of mate- 
rials and programs devel- 
oped, revised, or supported 

Number and degree level of 
students receiving 
stipends 



Increase in personnel 
qualified to educate 
minority handicapped 
children and youth 

Increase in minority 
personnel qualified to 
educate handicapped 
children and youth 



IS J 



Figure C-6. Minority 



COMPETITION OBJECTIVE 



INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



• To provide preservice 
doctoral and postdoctoral 
preparation of professional 
personnel to conduct train- 
ing of teacher trainers, 
researchers, administrators 
and other specialists 
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Legislati on and Regulations 



• P.L. 91-230, Sections 631, 
632, and 634, Part D, as 
amended by P.L. 93-380 
(1974), P.L. 9B-199 (1983) 

• 34 CFR Part 118 

• EDGAR 34 CFR Parts 74 , 75, 
77, and 78 

Funding (FY 86) 

• (See Table C-K) 

Eligibility Requlreatmt s 

• lHEs, public and Private 
colleges, other non- 
profit public and private 
agenc i es , inc lud ing 
parent organisations 

Conditions That Must B<? Met 

• All applicant agencies 
must meet state and pro- 
fessional standards. 

» Applicants must show 
evidence that proposed 
projects address state 
and /or nation.il needs <latn. 

Repo r t i nf^ Re qu i reroen t s 

• Annual Financial Status 
Report 

• Final Performance Report 
(within 90 days after 
project termination) 

• Annual Performance Report 
(continuation projects) 

Monitoring 

0 Grants are assigned to 
project officers to 
monitor by state. 

• Project officers do tele- 
phone monitoring of 1/3 
of grantees annually. 



Applications Funded <8*w. FY86) 

• (See Table Ol . ) 

Review Criteria 

• Extent of need (25 points) 

• Program content (15 points) 
m Operation plan (15 points) 

• Evaluation plan (15 points) 

• Quality of key personnel 
(15 points) 

• Adequacy of resources 
(5 points) 

m Budget (10 points) 

Review Process (FY 86) 

• Applications are reviewed by 
external peer review panels 
consisting of 3 reviewers* 

- panel limited to reviewers 
with a doctorate who are 
working with doctoral 
programs ; 

- panels should include 
1 repeat reviewer and 

1 director of a current 
federally-funded doctoral 
or postdoctoral program. 

• Project officers 1 recommen- 
dations are based heavily on 
raw and standardized score 
ratings by peer review 
panels and on panels' 
narrative justifications. 

• Review process usually takes 
3 months ur more. 



• Number, category , and level 
of students trained 

# Number, category, and level 
of students receiving 
stipends 



0 Increaae in doctoral level 
personnel qualified to 
train teacher trainers, 
do research, administer 
program 



Figure C-7. Leadership 



COMPETITION OBJECTIVE 



INPUTS 



T 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



• To support projects to 
develop, evaluate, and 
distribute new approaches 
for preservice and 
inservire training 
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Legislation and Regulat ions 

• pa. 91-230, Sections 631, 
632, and 634 f Part D, as 
amended by P.L. 93-180 
(1974), P.L. 98-199 (1983) 

• 34 CFR Part 318 

• EDGAR 34 CKR Parts /4 f 75, 
77, and 76 

Funding (FYB6) 

• (See Table C~l.) 

Eligibility Requirements 

• IHEs, public and private 
colleges, SEAs, LEAs, 
other nonprofit public 
and private agencies 

Conditions That Must Be Met 

• Applicants must show 
evidence that proposed 
projects address state 
and/or national needs 
data. 

Reporting Requirements 

• Annual Financial Status 
Report 

• Final Performance Report 
(within 90 days after 
project termination) 

• Annual Performance Report 
(continuation projects) 

Monitoring 

• Grants are assigned to 
project of ficers to monitor 
according to their exper- 
tise and their assigned 
states. 

• Project officers do tele- 
phone monitoring of 
grantees. 



Applications Funded {New, FY86) 

• (See Table C-l.) 

Review Criteria 

• Extent of need (25 points) 

• Program content (15 points) 

• Operation plan (15 points) 

• Evaluation plan (15 points) 

• Quality of key personnel 
(15 points) 

9 Adequacy of resources 
(5 points) 

• Budget (10 points) 

0 1 h ej__S e Ij^c^io n Cons id [era- 
t ions 

• Potential for broad impact 

• Potential to improve 
significantly the quality 
of personnel preparation 
efforts 

Review P rocess ( FY8 6 ) 

• Applications are reviewed by 
external peer review panels 
consisting of 3 reviewers* 

m Project officers* recommen- 
dations are based heavily on 
raw and standardized score 
ratings by peer review 
panels and on panels' 
narrative justification. 

« Review process usually takes 
3 months or more. 



Number, nature, and quality 
of model approaches for 
preservice training of 
regular educators 

Number, nature, and quality 
of model aiM'tifiiches for 
inservice u ining of spec- 
ial education personnel, 
including classroom aides, 
related services personnel, 
and regular education 
personnel 

Number, nature, and quality 
of materials developed 

Number and type of mate- 
rials, models, or other 
information disseminated 

Number and types of dis- 
semination targets 



Use of quality practices 
for preservice and in- 
service training of 
personnel (all levels) 
who serve handicapped 
children and youth 

Improved quality of 
personnel trained in 
these programs 



us 



Figure C-8. Special Projects 



COMPETITION OBJECTIVES 



INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



To provide preservice and 
lnservice training of 
regular v+'watvrn to assist 
with the . i«nt ifirat ion and 
delivery of services to 
handicapped children and 
youth 

To develop personnel 
training programs for 
regular educators on a 
statewide basis 

To disseminate/diffuse state* 
wide training, models 
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Leglslat Jon _and Re^uJat_ions 

• P,L. 91-210, Sections 631 , 
632, and 634, Part I\ as 
amended by 1\L. 91-380 
(1974), P.L, 9H-19V (1983; 

• 34 CFR Part U8 

• EDGAR 34 CFK Parts 74, 75, 
77. and 78 

F unding (FY 8b) 

• (See Table C-l.) 

Kllgi billty Requi rements 

• SEAs only 

Conditions That Must Be Met 

a Applications must show 
evidence from data col- 
let ted annually that need 
does exist, and reflect 
priorities identified in 
CSPD portion of state 
plans. 

Reporting Requirements 

a Annual Financial Status 
Report 

• Final Performance Report 
(within 90 days after 
project termination) 

a Annual Performance Report 
(continuation projects) 

Monitorin g 

a Grants are assigned to 
project officers to 
monitor by state, 

a Project officers do tele- 
phone monitoring of 
grantees. 



Applications Fund ed (New. FY86) 
a (Sec Table C-l.) 

Review Criteria 

a Extci.t of need (25 points) 

a Program content (15 points) 

a Operation plan (IS points) 

a Evaluation plan (15 points) 

a Quality of key personnel 
(15 points) 

a Adequacy of resources 
(5 points) 

a budget (10 points) 

Review Proress (FY86) 

a Applications are reviewed by at 
least 3 independent reviewers 
(1 of whom is nonfederal) , 
who then convene as a panel. 

a Project officers* recommen- 
dations are based heavily on 
raw and standardised score 
ratings by peer review 
panels and on panels 9 
narrative justification. 

a Review process usually takes 
1 months of mute* 



a Number and nature of state- 
wide training models 
developed/implemented to 
identify and work with 
handicapped students in 
"mains t reamed' 1 settings 

a Number and nature of "best 
practices" evident in these 
statewide training models 

a Number of regular educators 
trained in these programs 

a Competencies demonstrated 
by regular educators as a 
result ot such training 



a Increase in regular 
education personnel, 
including administrators 
and supervisors, who are 
qualified to provide 
services to handicapped 
children in "sains t reamed" 
settings 




Figure C-9. Regular Educators 
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COMPETITION OBJECTIVES 



INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



• To assist SEAs to increase 
the supply and improve the 
quality of teachers of 
handicapped children and 
youth ami their super- 
visors 

• To improve SEA leadership 
capability for development, 
dissemination, and incor- 
poration of successful 
educat ional training 
practices statewide 

• To assist states in pre- 
service and inservice 
training in areas of unique 
need 



Legislation and Regulation s 



• PL 91-230, Sections 631, 632, 
ami 634 y Part D. as amended by 
P.L. 93-380 (1974), P.L. 98-199 
(19»3) 

34 CFK Part 318 

• EDUAK, 34 C*H Parts 74, 7S f 77, 
and 78 

Funding (FY 86) 
(See Table C-l.) 

Eligi bility j%% ;u? regents 



• Only SEAs are eligible. 

• SEAs may conduct training direct 
ly or contract with IHEa. 

Conditions, That Must Be Jtet 

• Projects must deal with unique 
statewide training needs iden- 
tified by data collected annu- 
ally, and recommended by ':SPD 
unit . 

• Projects Bust focus on training 
of supervisors and teachers of 
handicapped children and youth* 

• Projects may include training in 
management and crganizat ional 
design to enhance states* ability 
to provide special education 
services to children snd youth. 

Reporting Requirements 



Applications Funded (Mew, FY86) 

• (See Table C-l.) 

Review Criteria 

• Extent of need (25 points) 

• Program content (15 points) 

• Operation plan (15 points) 

• Evaluation plan (15 points) 

• Quality of key personnel 
(15 points) 

• Adequacy of resources 
(5 points) 

• Budget (10 points) 

Review Process (FY86) 

• Applications are reviewed by 
panel of external and internal 
peer reviewers* current SE*» 
employees, former SEA employ- 
t?es, Individuals who current- 
ly or formerly work closely 
with SEAs. 

• Project officers* recommenda- 
tions are based heavily on raw 
and standardised score ratings 
by peer review panels and on 
panels' narrative Justifica- 
tions. 

• Review process usually take? 
J months or more. 



• Annual Financial Status Report 

• Final Performance Report (within 
90 days of termination) 

• Annual Performance Report 
(continuation projects) 

Monitoring 



a Grants are assigned to project 
officers to monitor by state. 
Project officers monitor a mini* 
of 10 projects annually. 



• Monitoring is conducted by phone 



• Number and category of 
personnel trained (all 
levels) 

• Number and nature of 
training programs devel- 
oped and disseminated 

• Number and nature of 
"bust practices 1 * evident 
in these statewide 
training programs 

• Number and nature of 
statewide priorities 
addressed by these 
training programs 



• Increased and improved 
capacity of states to 
train teachers and 
supervisors 



1 oo 




fig u re -£-40— Mh ? r »Jfc« 
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INPUTS 



IMPLEMENTATION 



OUTPUTS 



OUTCOMES 



• To provide training and 
information to parents of 
handicapped children, and 
to persons win* w^rk 
with parents 
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Leg! al at Ion and Regulat ions 

• P.L. 91-230, Sections 631, 
632, and 634, Part 1) as 
amended by P.L. 91-380 
(1974), P. I, 98-i« p '1981) 

• 34 L>* Part 1J« 

• EDGAR, 14 IKK Tar is 74, 7*) 
77, and 7H 

Funding (FY 8b) 

• (See Table C-l.) 

EJ ijjj b illty Requi rements 

a Parent organizations or 
private, nonprofit agencies 
seating very specific 
requirements in the law 
that ensure parents a male 
role in developing and 
Implement ing proposed 
projects 

8euj>rt in^ Requirement* 

9 Anno a 1 Financial Stat on 
Keport 

• Final Performance He port 
(within W days ol 

if rmin.it ion) 

• Annual Performance Keport 
(iont inuat ion projects) 

Monltorijijj 

m Grants are assigned to 
project officers to 
monitor by state. 

• Project officers do tele* 
phone monitoring of 
grantee s * 



Applications Funded (New, FY 8b) 

• (See Table C-l.) 

Rev lev Criter ia 

• Extent of need (23 points) 

• Program content {15 points) 

• Operation plan (15 points) 

m Evaluation plan (15 points) 

a Quality of key personnel 
(15 points) 

m Mequacy of resources 
i points) 

• Budget (10 points) 

L t he r_ Selecti on Consider s I i ons 

• (Geographical distribution 

• For us on providing service to 
parents of all handicapped 
children in all categories 

• Focus on unique training and 
information needs of parents 

Review Pro cess (KY86) 

• Applications arc reviewed by 
external peer review panels 
consisting of 3 reviewers. 

m Project officers* recommenda- 
tions arc based heavily on 
and standardized score ratings 
by peer review panels and on 
panels' narrative Justifica- 
tion. 

• Review process usually takes 
3 months or more. 



Types of information 
provided to parents 

Number and types of cur- 
riculum and programs 
de ve 1 oped / rev i sed 

Number %nd nature of parent 
training at t ivit ies conducted 

Competencies and knowledge 
demonstrated by part lei* 
pants in parent projects, 
e.g. : 

- handicapped children's 
educational rights; 

- educational system*** 
responsib 1 1 i t i cs ; and 

- effective strategies 
for working with ednca- 
t ionai system. 



Figure C-ll- Parent Organization Projects 



Parents, and persons 
who work with trua, become 
actively involved an 
behalf cf their cMldrcn's 
education and assist in 
that process* 
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APPENDIX D 
The Study Sample 



• Table D-l. 



• Bar ' c Descr 
the Sample 

Table D- 2. 
Table D- 3. 
Table D- 4. 
Table D- 5. 
Table D- 6. 
Table D- 7. 
Table D- 8. 
Table D- 9. 
Table D-10. 
Table D-ll. 
Table D-12. 



Reasons for Deletions from and 
Additions to the Sample 

iptive Information for Projects in 



Special Educators" Projects (N=9) 

Related Services" Projects (N=5) 

Rural" Projects (N=5) 

Infant" Projects (N=3) 

Transition" Projects (N=5) 

Minority" Projects (N=3) 

Leadership" Projects (N=6)* 

Special Projects" (N=7) 

Regular Educators" Projects (N=4) 

SEA Projects" (N=4) 

Parent Organization Projects" <N=5) 



* One of the 7 projects in the Leadership subsample 
was dropped because available information was too 
minimal to include it in subsequent analyses. 
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Table D-l 

Reasons for Dele t ions from and Additions to 
the Sample for the Personnel Preparation Program Goal Evaluation 



Competition Area 

Special Educators 

Related Services : 
• "Paraprofessional" 



• "Career Ed" 

• "OT, PT, Nurse" 

• "Counselor" 

• "Therapeutic Ree." 



Total # 
in Sample 



(5) 
1 



1 
1 
1 
1 



Deletions/Additions 
No changes 



Delete PR029FH40077 
"Train. ..Deaf Interpreters" 



Add PRO29FH40014 

"Train. . .Paraprof essionals" 

No changes 

No changes 

No changes 

No changes 



Reason 



"Deaf interpreters" a much more spec- 
ific target (personnel) than the 
"typical paraprofessional" so not a 
true specimen of this subsample. 

Identified by random draw as 
"Replacement #1." 



Rural 
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Infant 
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Delete PRO29AH60O64 
Delete PR029AH60224 
Delete PRO29AH60O22 

Add PRO29VH60007 
Add PRO29VH60004 
Add PR029VH60003 

Delete PR029AH60O43 
Delete PR029AH60097 

Add PR029AH60060 
Add PR029AH60105 



Not a true specimen of focus on 
Rural training. 

Had to go to FY85 to complete sample 
with 3 projects that focused on RURAL 
training. 

Second project directed by same person. 
Early childhood focus, NOT infant. 

Next in random draw for replacements, 
with true focus on INFANT training. 
Two other replacement candidates dis- 
carded because of focus on early 
childhood, not infant. 
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Table D-l (continued) 



Reasons for Deletions from and Additions to 
the Sample for the Personnel Preparation Program Goal Evaluation 



Competition Area 
Transition 



Total # 
in Sample 



Deletions/Additions 



Delete PRO29VH60O05 
Delete PR029VH60O07 

Add PRO29AH60O26 
Add PR029AH60057 



Reason 

Anomalous selection » 

Second project with same director. 

Next in random draw for replacements, 



Minority 
Leadership 
Special Projects 



3 
7 
7 



No changes 



No changes 



Delete PRO29KH40141 



Add PR029KH40016 



Emphasis on delivery of training; 
not much development. 

First selected by random draw. 



SEA Projects 



Regular Educators 



4 
4 



Add CALIFORNIA 
Add NEW YORK 



Only 3 states had both SEA projects 
and Regular Educators projects 
(KS, MN, SD). Therefore, added the 
4th project of each type from two 
big states. 



Parent Organization 
Projects 



No changes 
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Table D-2. "Special Educators" Projects (N-9) 



Title 



Target Roiea 



"Hultidiscipl inary Master 's Degree 
Program for Teachers of the Behavior* 
ally Disordered/Emotionally Disturbed" 

Type uf Institution: I!|£ 

Location: Maryland 



Teacher of behaviorally/emot tonally 
disturbed* especially secondary ageu 
encarcerated youth 

Level : M. A. ; Preservice 



Target handicapped Population 

Bthavi orally disordered, emotionally 
disturbed children and youth 



FY84 
KY85 
JY86 



Funding 

$89, 
$89, 'J 'Hi 
$89,310 



"Bachelor's Degree Teacher Training 
Program in Ueaf Education" 



Type ot institution: INK 
i.ncat ion : Arkansas 



Certified special education teachers 
of hearing impaired 



Level: B.A. *>r Certificate; 
Preservice 



Ma inst reamed imdcrserved , transition, FY 84 $45, 000 

tarly childhood, institutionalized, *Y85 $45,001) 

and multiply handicapped deaf and FY8f> $45,000 
hearing impaired children and youth 



"Pre-Service Training of Master's 
Degree Students in the Area of 
Adapted Physical Edurat ion" 

Type uf Institution: 1HK 

Location; Nevada 



Certified teacher in adapted physical 
educat ion 



Level; M.A.; Pri-sct v i i 



Handicapped children and yo Uh in 
special schools, regular schools, 
and itinerant programs 



FY84 
FY8b 
FY86 



$25,7u9 
S2b f 769 
$^5,7*9 



"Personnel Preparation in the Area 
of Behavior Disorders" 



Type of inttitution: 1HE 
Local ion: Utah 



Classroom teacher, .hildrer. and youth 



Level: B* A. and post 8. A.; 
Preservice 



Behavioral ly disordered children 
and youth 



FY84 
rY8h 



$78,794 
$78,794 
$78,794 



"Preparation of Special Educators 
Undergraduate Program in Applied 
Developmental Disabilities" 

Type ut institut e; HIE 

ideation: Florida 



Classroom teacher 



U*v*l : B. A. ; Preservice 



Learning disabled, emotionally 
disturbed, and mentally retarded 
children and youth 



FY84 
FY8S 
FYHb 



$b7,14M 
$b7. *49 
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Table !>-2. "Special 



Title Target Roles 

"Waster's levt*l Training lor Teachers Special cducat inn teacher 

of (hi* Handicapped" 

Type of Institution: IHF. Level: B.A.; Preservire 

Location: Orison 



"Training of Speech/ Language Patholn- Speech/ language pathologists, 

gists. Audiologists audinlogists 



Type of Institution: IHK Level: H. A. ; Preservlci- 

ideation: New York 



"Preparation of Special Kducaturs for Special education teacher* 

the Mlldly/Hodi-ratelv Handicapped" (all levels) 



Type of Institution: IHK 
Locat ion: Nehr.i.sk.j 

"Training Specialists to Appropri- 
ately Serve Blind and Visually 
Impaired Individuals" 

Type of Institution: IM»1 

Location: California 



Leve I : I ert if icate ( ^endorse- 
ment ) ; Vrc service 



leachers of visually h.mdicappcd 
children and vouth (ages 0-21) 



Level: Certificate; A. ; 

Preservice, lnserviee 



Projects (continued) 



Target Handicapped Population Funding 

Handicapped children, youth, and young FY 8$ $7« f 575 

adults FW* $7*#,S^-> 

KY86 $79,575 



Children and adolescents severely FY84 $21.3** 1 

handicapped by common icat inn PY85 $21,l*n 

disorder* l'VHn $21, W» 



Mi Idly/moderatelv handicapped children FY84 $<t7,82 7 
and youth (ages 0-21) in rural areas FY85 $52,917 
(i.e., those identified under the FY 8b $35,584 

categories of specific learning dis- 
ability v behavioral ly impaired, or 
rducable pentallv handicapped and who 
are ma instr canned into regular educa- 
t U>n) 



Visually handicapped children 
youth (ages 0-21) 



and FY84 $76,805 

FY85 $7h.805 
FY86 $76,805 
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Table D-3. "Related 



Title 

"Pioject FAkA: A Project to Improve 
the Training and Utilization of 
Paraprofessionals in Special ^dmvit ion" 

Type of Inn; i tut ion: I Hi: 
ideation: Nebraska 



Target Roles 



Special educators (primarily class- 
room teachers in public ant! private 
i nst i tut ions > ; spuria! education 
paraprofeNKion.»1 s 

Level: Cert if irate of completion; 
Preservice 



"Occupational Therapy Students* 
Training tor High School -Based 
Independent Living Skills Development" 

Type of institution: 1HK 

Location: CalMnrnia 



Hi- « up.it ioiial therapists in school - 
based programs 

Level: M,A. ; Preservice 



"Training Par%»nt -Trainers to Facilitate 
Career Situation Practices in the 
Homes of lit ban Handicapped Youth 1 ' 

Type of institution: I 111, 

Loot ion: Ni-w York 



Career education trainer e' parents 
(primari >y school personnel ) 

level: Cert i f irate of competency; 
Preservice 



M t:allaudet College Project lor the 
Training of School Counselors lor 
Hearing Impaired Children" 



School counselors of hear! ng impai red 
children and vout h In a v.irlelv of 
settings (all levels, public and 
private) 



Type of Institution: I UK level: M.A, ; Preservice 

Location: Pi strict of Columbia 



"Preservire BA/MA Level Programs in 
Therapeutic Kecreat ion/Chi Id Liic 
Services" 



Type of Institution: 1HK 
Location: New York 



Therapeutic recreation/child life 
specialists and adir listralors in 
pediatric units of general or spec 
ialty hospitals 

Level: B.A. ; M.£.; Preservice 
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\" Projects (N«5) 



Target Handicapped Population Funding 



School-age children and youth (all FY84 $72,640 

conditions, all levels) FY85 $72,640 

WHb $72,640 



Severely handicapped high school KYS4 $94,860 

students FYM $94,860 

FY 86 $94,860 



lnner-iitv handicapped Hispanic FY84 $8b,889 

students (primarily neurological ly KY85 $86,889 

and/or emotionally handicapped) FY86 $86,884 



Hearing impaired children and vouth KY84 $14,967 

FYS') $34,967 
FY8n $41,659 



111 and handi rapped children, inrlud- FV84 $44,907 
ing infants, toddlers, youth, and KY85 $44,907 

adolescents W86 $44,907 
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Table 0-4, "Rural" Projects <N«5) 



Tilli 



"Preservire Training of Rural Special 
Education Personnel in tin* Area of 
Severely Handicapped" 

Type of Institution: 1HK 

location: Vermont 

"Kural I3en*»ral Special Ktforat Ion 
Preservice Personnel Preparation 
Prog ran with Emphasis on Low- 
Jncidence" 

Type of Inst it tit ion: INK 
Location: Oregon 

"Preparation of Special Educators: 
Bachelor and Masters Level Training 
for Teachers of the Hearing Impaired 
in Rural Kentucky" 

Type of Institution: IHK 

Location: Kentucky 



"Preparation of Special Educators: 
Masters Level Training for Teachers 
of the Communicatively Handicapped 
in Rural School Districts" 

Type of Institution: ME 

Location: Mississippi 



Target Holes 



Target Hai dirapped Population 



Funding 



Classroom teacher, teacher consultant, Rural moderately to severely/ 

resource room teacher, itinerant profoundly handicapped students 

teacher and families 

Level: B. A. ; Preservice 



EY84 $19,852 
KY85 $4 1,009 
FY 86 not in grant file 



Rural education consultant, rural 
special education gene rail HI 



i^vcl: M.A.; Preservice 



I,ov Incidence handicapped learners 



c:lansroom teacher in resident school, 
self-contained classroom, or 
itinerant program 



Level: R.A. , W.A. ; Preservice 



Deaf children 



Rural service positions, speech - 
1 anguagc pathologi s t s 



Level: M.A.; Preservice 



Communicat ively handicapped in 
rural school districts 



"Preparation of Karly Childhood 
Special Education Rural Service 
Providers" 



hirert service providers, indirect 
resource managers 



Young handicapped rural children 
three years and above 



FY84 



$78,840 
$78,840 
$78,840 



FY84 $25,629 
FY85 $18,123 
FY86 $50,673 



FY84 $75,000 
FY85 $80,000 
KY86 $83,549 



KY84 $52,968 
FY85 $52,968 
FY86 $52,968 



Type of Institution: I HE 



Level: H. A> ; Preservice 



Location: Washington 



Table 0-5. "Infant" Projects (N«l) 



Title 

"Preparation Program for Parent - 
Infant Education Specialists" 



Type of Institution: I UK 
Local ion: Washington, ILC- 



Target Holes 



Parent Educator and Consultant 



Level : M. A. ; Preservice 



Target Handicapped Population 



Handicapped, oult i-handi capped 
infants three years and below, 
minority and tinderserved, espec- 
ially hearing Impaired and their 
parent a 



FY84 
FY85 
FY8b 



Funding 



$72,507 
$74,721 
$72,502 



"Preparation t>i Special Educators, 
Master's Level Training for Teachers 
of Handicapped In I ants and Pre-School 
Children" 

Type of Inst i t ut ion : I MK 
Location: Pennsylvania 



Karly chi Idhuod spec ial educators, 
early inter vent lonists 



Levi- J : M. A, ; Preservice 



Handicapped inlauts and pre- 
schoolers 



FY84 
FY 85 
FY86 



$54,706 
$54,706 
$54,70f> 



"Preparation of Special Kduca'.ors: 
Frcservire Training oi Master's 
Level Speech- language Pathologists, 
Audiologists to Serve Infants/ 
Preschool Chldren" 

Type of Inst i tut Jon: IHK 

Location: South Carolina 



Audiologists, speech- language 
pathologists; early interventionists 



Level: M,A.; Preservice 



Handicapped infants: 
five years 



birth to 



FY84 
KY85 
FY86 



$47,787 
$42,742 
$41,2*0 
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Table "Transition" Project* 



Titl< 



Target Holes 



Target Handicapped Population 



Funding 



"Masters level Training fur Spuria! 
Vocational Kducators o! the Handi- 
capped" 



Type of Institution: 1 UK 
I a) rati on : Don tie c t i c u t 



Secondary- level vocational classroom 
teachers/ instructors in nontradlt Ional 
settings; supervisors and administra- 
tors of vocational programs 



level: H.A.; Preservice 



All age groups, educational levels, FY 84 $44,054 

and types of handicapping conditions FYS 5 §44,054 

(who typically receive prevocational — FY86 $44,054 
1 3-15 — and vocat ional /career/occupa- 
tional training— 15-21) 



"Preservice Training of Special Educa- 
tors, Counselors, and Vocation;:! 
Educators to Work Cooperatively in 
the Vocational Programming of H.ijuH- 
capped Students" 



Type of Institution: I UK 
Location: Connect Scut 



Secondary- level personnel in Indus- 
trial Arts, Vocational Education, 
Special Kthieat ion, School Counsel ing 
(e.g., industrial arts instructor, 
vocational education instructor, 
special education resource specialist 
srhonj counselor) who work in a 
variety of settings (e.g., local 
school s , regional vocat lon;i 1 - tec tin i- 
cal schools, sheltered workshops, 
other human services agencies) 

level: Certificate; M.S.; 
Pre service 



Secondary level handicapped students , 
crossing all conditions 



FY 84 
H85 
FY86 



$45,000 
$50,000 
$57,000 



"Transf t Ion Special Mutators for 
Seriously Emotionally Disturbed 
Adolescents: An Education Specialist's 
Degree level Training Program" 



Type of Institution: 1HK 
location; District ol iiojurohi.i 



Transition special educators In a 
variety of settings (e.g. , lFAs f 
private psychiatric facilities, 
community school s y resident ial/dav/ 
liospi t al programs) 

level : Kd.S. ; Preservice 



Seriously emotionally disturbed 
adolescents 



FY84 $7«,054 
H85 $79,054 
KY8f> $80, 90 7 



"Secondary Vocational Kduc.it ion for 
the Handicapped" 



Type of Institution: 

_ „^„Locat i on : ind i ana 
LKIL 

-TfO 



1HK 



Secondary special educator with 
ability to provide comprehensive 
vocat ional /career education; resource 
consultant to vocational educators; 
special educator with ability to 
adapt curriculum to needs of secon- 
dary /post secondary handicapped 
learners (in middle, bight and alter- 
native schools and postsecondary 
schools) 



level; Certificate; B.A. ; M- A. ; 
Preservice 



Youth at secondary /post secondary FY84 $100,000 

level who are learning disabled, KY85 $100,000 

mentally retarded, seriously emotion- FY 86 $100,000 
ally disturbed, physically handicapped, 
severely/profoundly handicapped 
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Table "Transit ion" Projects (continued) 



Title 



Target Holes 



Target Hand i rapped Population 



Funding 



"Leadership Development Program in 
Vocational Special Needs KUm.it ion* 



Local leadership personnel in voca- 
tional special education in a variety 
of agencies (e.g,, schools, rehab i li- 
tat ion far! I it ies, correct ional 
tenters, etc.) which all receive 
federal funding under Pi. 94-1 Ul and 
PL 



All handicapped populations served 

hy LEAs in Illinois (youth and adults) 



KYR4 
FY 8 5 
KYHb 



$72,300 
$72,500 
$72,500 



Type of Institution: 1 lit: 
Local ion: P i itioi 



I.t-Vcl: 4 units of credit toward M.A. 
nr doctoral device; 
I'resrrviee 
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Table D-7. "Minority" Project* (N-3) 



Title 



"Preservlie Special Fducation 
Preparation ol Minority Status 
Teachers" 

Type of Inst itut ion: IKK 
Location: Iowa 



Target Roles 
Classroom teacher 

level : H.A. ; Preservice 



Target Handicapped Population 



learning dibbled or emotionally 
disabled children 



Funding 



FY84 $b3 f 608 
FY85 $o4,220 
FY 8b $bb f 989 



"Bilingual Speeial Kdtu .it inn 
Administrat ive Interns Pioject" 



Type of Institution: I.KA 
Locat i on : ialii orn i a 



Building administrators (principals 
and assistant principals) 



Level: M.A. ; Preservlee 



Kcgular, special education, bilingual, 
linguistically and culturally diHer- 
en t student populations 



FY84 $4 7 , 00 3 
FV8% $70,<M)O 
FV86 $70,000 



"Preparation of Teachers to Serve 
Severely Behavioral! v Disordered 
Students from Minor i tv Populations 

Tvpe of Inst it nt ion? Private* 

nnuprof it 



Classroom teacher 



level: MtA.; Preservice 



Behavioral ly disordered and severely 
behavioral ly disordered minority 
vh i Idren and youth 



H84 $84,2^1 J 
FY8S $84/241 
FY8h $84 ,.>41 



U.cat ion: nhio 



ERIC 
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Tabic 0-8. ''Leadership Projects" (ta~b) 



Title 



H Pn?pflr.itiun of Special Fducalion 
Personnel" 



Type of Institution: HIK 
Loral Ion : Kent ucky 



Target Roles 

Special education teacher trainers, 
researchers, adrainist rators 



Uwl: Itoc i oral (Ed.l).); 
Preservice 



Target Handicapped Population 



Handicapped children, youth* and 
adult a 



Funding 



FY84 $103,748 
FYBS $103,748 
FV86 $103,795 



"Doctoral Training in Special 
Education/School Psychology and 
the Handicapped Hispanic Id" 



Type uf Institution: INK 



Special education/school psychology 
teacher trainers, researchers, 
special isis, school psychologists, 
administrators, supervi >rs 

Level: Doctoral (Ph.D.); 
Preservice 



Handicapped Hispanic children and 
youth with learning and behavioral 
problems; children with behavior 
problems 



F¥84 $110,211 
KY85 $110,211 
H86 $110*899 



Location: Texas 



"Doctoral Leadership Training in 
Teaching and Ke sean-h-- Psychol ingnis- 
lies of Earlv Childhood Language 
Disorders from a Hi lingua! and Multi- 
cultural Perspective" 

Type *>J lin-l i t tit ion: MIF 

Local ion: Massachusetts 



Pn i vers ity-1 eve 1 teacher trainers and 
researchers ; admin ist ratnrs * evalua- 
tors, consultants, clinicians 



U*v«*l : iHM'toral ; Preservice 



Young children with language and 
cwninun i cat ion disorders, who come 
f rom bi 1 ingual , minority* and 
multicultural homes 



KY84 $61,764 
$61,764 
KY86 $75*8 I >1 



"Preparation oi leadership Personnel 
in t'ommimi cat ion Disorders" 



Type of Inst it tit ion: IHK 
Location: Washington 



College and university tea* her 
I miners of (.omsmmi cat ion Disorder 
Special isis (di rect clinical service 
provide is) 



Leve!; I\>ctoral; Preservice 



Preschool and school-age children 
with communirat ton (speech-hearing) 
handicaps, focusing on children 
with cotJBBunicat ion disorders in 
tlie early childhood* severely handi- 
capped, and hearing' impaired 
popular ions 



KY84 $101*712 
FY85 $101,732 
FY86 $'03,712 



21B 
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Table I>-8. "Leadership Projects" (continued) 



Title 



"Preparation of leadership Personnel 
for the Visually Impaired" 



Type nt Institution; IMK 
Location: Tennessee 



Target Rolen 



Ter*c»ei trainers in colleges and 
roiversit ies preparing personnel in 
the special education of visually- 
impaired students; researchers, 
administrators, supervisors in LRAs, 
SKAs, regional and other programs; 
consul tants 

Level: Doctoral; Preservice 



Target Handicapped Population 



Funding 



Visual ly-iropai red and multiply 
handicapped children and youth 



FY84 
FY85 
FY8h 



$87,474 
$90,474 
$98,000 



"George Mason University Dot* tor <>f 
Arts in Education: Special Kducat Ion" 



Type ot institution: 1 HE 
Local Ion: \ I rginia 



Special education administrators, 
teacher trainers, rorr iculnm special- 
ists, dingnost it i,« researchers ; 
pul irymakers at hu.jl, state, federal, 
i n t ernat i oiia J I eve 1 s 

Level: Doctoral (D»A.Kd); Preservice 



All handicapped learners 



K84 $101,469 
KY85 $ 93,169 
FY8b $ 9), 169 



2.b 
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Table D-9. "Special Projerts" (N-7) 



Tit It 



Target Rules 



Target handicapped Population 



Funding 



•'Preparation of Speech Pathologists 
and Audiologists (to nerve) Handicap- 
ped Children, Birth to Five" 

Type of Inst i tut ion: 1IIK 

Location: Colorado 



"Training System for In I ant Service 
Providers" 



Type of Institution: IMF 
Location: District of Columbia 



Speech pathologist*, audiologists 



Level; M, A» (Specialists); 
Preservice 



Medical, educational, and thera- 
peutic specialists in hospitals and 
in the coosnunity who serve handicap- 
ped infants in their first year nf 
life 

Level : Special ists, inservice, 
P re service 



Young children (0-5 years) with com- 
munication disorders 



Sick, high-risk, handieapped infants*, 
birth to one Year old 



FY84 
FY85 
FY86 



FY84 

FY85 
FY86 



$63,111 
$60,231 



$100,000 
$122,332 
$ 95,699 



"Training School and Community 
;;^»*vire Personnel tor Transition ol 
Secordary Handicapped Students to 
Post ehoo 1 F.nv i r onmen t s" 

Type" Institution; IMF 

Location: Oregon 



"Graduate Training in Technology 
for the Handicapped" 



Type of Institution: IMK 
Lncat ion: Maryland 



"Advancing Technology Through 
Statewide Recreation Network" 



Type of institution: I UK 
Location: Massachusetts 



ERLC 
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Serial education teachers, school 
administiators. and community 
service providers 



Level : Inservice 



Administrators, special educators, 
therapists, and computer coordinator 
(emerging role) 

Level: M.S. or Certificate 



Therapeutic recreation and other 
specialists, educators, agency 
administrators , parents 

Level: Inservice, Preservice 
(graduate level) 



Moderate to severely handicapped 
youth (middle school and high school 
age, chronologically speaking) 



All; multiple-handicapped individ- 
uals most frequently represented 



FY84 
FY85 
FY86 



FY84 
FY85 
FY86 



$63,113 
$75,199 
$73,81 J 



All disabilities 



$44,705 
$63,111 
$60,251 



KY84 $74,561 
FY85 $74,563 
FY 86 $80,000 
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Table "Special Projects" (continued) 



Title 



"Ha ins t reaming: Home/School 
Integrated Intervention*' 



Target Roles 



Special educators, administrators, 
parents, and support personnel 



Target Handicapped Population 



Mildly handicapped and "at-risk" 
students in elementary and secondary 
schools 



Funding 



FY84 $?7,846 
nT85 $S9,810 
FY86 $84,391 



Type of Institution: Area Education 

Agency 

Location: Iowa 



"Project S.H A.R.E.' 



Level: Inservice 



Information givers, disabled Pre- 
senters, and volunteer facilitators/ 
trainers 



Type of Institution: Parent (community- Level: N/A 

based coalition of 
parent organisa- 
tions) 



All who attend public schools 



FY85 $92,660 
FV86 592,660 



Location: Illinois 
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Table D-10. "Regular Educators" Projects (N~4) 



TUit* 



"Preparation »i Regular and 
Special Educators" 

Type of Institution: SEA 

Location: South Dakota 



Target Roles 



Kegular fdwators, special educators 



Level : Inserviee 



Target Handicapped Population 



Funding 



Mildly handicapped students in regular KY85 $44,576 
class rooms FY86 $5 5 , 505 



"Prereferral to Special Education 
Model" 



Typo of Institution: SKA 
Location: Minnesota 



Kegular educators, special educators 



Level : In-service 



learning disabled and nil wildly FYB5 $69,518 

handicapped students In public schools, FY 8b $69,518 
including children with behavior 
problems 



"School Admin 1st ratnrs Statewide 
Inservice" 



Type of Institution: SKA 
location: Kansas 



School administrators, superintendents, 
school board members 



Leve 1 : I use rv Ice 



All handicapped students in regular 
r] ass rooms 



KY84 $44,652 
KY85 $44,b52 
FY86 $44,652 



"Expanding Kdncal ional Opportu- 
nities lor LI) Students" 



Regular class mom teachers, building 
prini ipals, school psychologists, 
special educators , 1 UK tear he r t r.i incrs 



Learning disabled students and "those FY85 $110,000 
not classified who are experiencing KY86 $122,000 

learning difficulties" 



Type oi Institution: SKA 
Locat ion: New York 



Level 



Inservice 
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Table D-ll. "SEA Projects" (N-4) 



Title 



f, ED-WD: Statewide Inservice Training 
Models for Increasing Col laborat ive 
Interaction between Physicians and 
Educators to Improve Services to Handi- 
capped Children — Provide Training for 
Educators and Physicians" 

Type of Institution: SKA 

Location: Cal i lornia 



"A Statewide Network tor Special 
Education Pa raprof ess iona I Preserviee 
ami Inservice Tra in in ft" 

Type of Inst i tut inn ; SKA 

Location: Kansas 



"Preparing Special Educators for 
Partnerships in Special Needs Voca- 
tional Training" 

Type of Institution: SKA 

Location: South bakota 



"Regional Inservice/Preservice in 
Early Childhood Handicapped, Transi- 
tion to Work, Limited English Pro- 
ficiency (t.EP), and Indian Social 
Work Aide (1SWA) M 



Type of Institution: SKA 
Location: Minnesota 



Target Roles 



Target Handicapped Population 



Funding 



Primary care physicians (develop- 
mental /behavioral pediatrics* family 
practice, general practice); special 
educators and regular educators 
(administrators, teachers, school 
nurses, other support personnel) 

Level: Inservice; Preservice 



Handicapped children in general — 
0-5 years and school age 



FY84 $85,000 
FY85 $85,000 
FY*6 $85,000 



Paraprofessionals in all areas of 
special education, all settings, 
all lewis 



All conditions, ages 0-21 



FY84 $109,710 
KY85 $109,710 
H86 $109,710 



Level: 



Pa raprof ess ional Permit at 
Level 1, 2, and 1; Inservice; 
Preservice 



Special education teachers, some of 
whom are trained through the project 
as "job development specialists" 

Level: Inservice 



Secondary 
youth 



-level mildly handicapped 



FY8A $49,076 
FY85 $76,664 
FY 86 (not in grant file) 



Special odurators, special education 
administrators, regular educators, 
community service providers 



All conditions, <or the following 
groups: children 0-3 years old; 
children and youth who art? Native 
American, or who are enrolled in 
secondary-level special education, 
or who have limited English 
proficiency (LEP) 



FY86 $80,000 
FY85 $96,782 
FV86 $96,782 



level : Inservice 



erJc 
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Table D-12. "Parent Organization Projects" (N-5) 



Tit li? 



Target Holes 



Target Handicapped Population 



Funding 



"Technical Assistance for Parent 
Programs - TAPP" (pritm* contract) 

Type of Institution: Private* 

nonprof it 



Effective Parent Training Project 



All currently funded parent organiza- 
tion projects under DPP for parents 
of children with all handicapping 
conditions 



Period 1 $129,723 

FY84 Option 1 $732,571 

FY85 Option 2 $750,868 

FY86 Option 3 $679,283 



Location: Massachusetts 



M Technical Assistance for Parent 
Program* - TAPP" (regional sub- 
contract) 



Effective parent training project 



All currently funded parent organ iza- FY84 $55,000 

tion projects under DPP for parents KY85 $58,000 

of children with all handicapping FY86 not in grant file 

I'ondi f inns 



Type of institution: Private, 

nonproi 1 t 

Location: New Hampshire 



"Nevada Specially Trained Effective 
Parents Project" 



Parent Yacil i tat or 



All handicapping conditions 



FY 84 $88,500 
FY85 $88,500 
FY86 not in grant file 



Type of Institution: Private, 

nonprof it 

I.oc a t i on : Ne vada 



"Parents Training Parents" 



Parent trainer 



All handicapping conditions 



FY84 $129,602 
FY85 $136,082 
FY86 $142,886 



Type of institution: Private, 

nonprof it 

Location; Michigan 



"Training Parents of Handicapped 
Children in Central Arkansas 



Type of Institution: Private, 

nonprof it 
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Location: Arkansas 

99ft 



Parent trainer 



Parents of children with all 
handicapping conditions 



BEST COPY AVAILABLE 



FV84 $42,225 

FV85 $42,225 

FY86 $105,225 (The extra 

funds cover a Joint grant 

in another city.) 
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Table E-l. List of Persons Interviewed 



OSERS/OSEP PERSONNEL 

Max Mueller 
Director 

Division of Personnel Preparation 

Harvey Liebergott 

Chief* Related Personnel Branch 

Division of Personnel Preparation 

Martha Bokee 

Manager , Related Services 

Personnel Competition 
Division of Personnel Preparation 

Frank King 

Manager, SEA Competition 
Division of Personnel Preparation 

Betty Baker 

Manager, Transition Competition 
Division of Personnel Preparation 

Jack Tringo 

Manager, Parent Organization 

Projects Competition 
Division of Personnel Preparation 

Norm Howe 

Chief, Leadership Personnel Branch 
Division of Personnel Preparation 



William "Pete" Peterson 
Manager, Special Projects 

Competition 
Division of Personnel Preparation 

Doris Sutherland 

Manager, Leadership Competition 

Division of Personnel Preparation 



CONGRESSIONAL SOURCES 

Gray Garwood 
Staff Director 

House Select Subcommittee on 
Education 

Dave Esquith 

House Select Subcommittee on 
Education 

Lani Florian 
Legislative Assistant 
Senate Subcommittee on the 
Handicapped 

STATE SOURCES 

Karl Murray 

Administrator, Personnel 

Development Unit 
California State Department of Education 

OTHER OSERS PERSONNEL 

Mike Herrell 

Office of the Assistant Secretary 
OSERS 

Richard Helia 

Project Officer, National Insti- 
tute of Handicapped Research 
OSERS 
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OSKKS/OSBP PKRSOHHKL (COHT.) 



OTHER DEPARTMEHT OF EDUCATIOM PSRSOWHSL 



Robert (Bob) Gi lino re 
Manager, Rural Competition 
Division of Personnel Preparation 



Dick Champion 

Manager, Regular Educators 

Competition 
Division of Personnel Preparation 

Angela Thomas 

Staff, Leadership Personnel Branch 
Division of Personnel Preparation 

Don B lodge tt 

Manager, Special Educators 

Competition 
Division of Personnel Preparation 

Sandra Hazen 

Manager, Infants Competition 
Division of Personnel Preparation 

Vickie Ware 

Menager, Minority Competition 
Division of Personnel Preparation 

Tom Finch 

Chief, Early Childhood Branch 
Division of Educational Services 

Marty Kaufman 
Director 

Division of Innovation and 
Development 

Paul Ackerman 
Acting Director 
Division of Program Analysis 
and Planning 

Etta Waugn 

Staff, Regional Resource Centers 
Division of Assistance to States 



Greg Frane 
Budget Analyst 

Office of Planning, Budget, and 
Evaluation 

Ann Weinheimer, Mar> Tapageorgiou 
Center for Statistics 

Larry La Moure 
Center for Statistics 

OTHER FEDERAL OFFICES 

James Ricciuti 
Budget Examiner 

Office of Management and Budget 
TAPP 

Martha Ziegler 

Director, Technical Assistance Center 
for Parent Organization Projects 

COMSTITUEHCIBS 

Judy Smith-Devi;; 
Private Consultant 

Dick Schofer 
University of Missouri 

Deborah Smith 
University of Hew Mexico 

Stan Dublinski 
American Speech and Hearing 
Association (AS HA) 
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COHSTITUEHCI ES CCOHT. ) 



mm 



Dave Bostatter 

Chief, Monitoring Branch 

Division of Assistance to States 

Bill TyrfcU 
Deputy Director 

Division of assistance to States 
Jim Hamilton 

Project Officer, Besearch Projects 

Branch 
Division of Innovation 

and Development 

Bill Wolf 

Acting Branch Chief /Project Officer 
Program Planning and Information 
Branch 

Division of Program Analysis and 
Planning 



Patty Guard 
Acting Director 
OSEP 



Doris Helge 

Western Washington University 

Oliver (Leon) Hurley 
Georgia State University 
Higher Education Consortium for Special 
Education (HSCSS) 

Herbert Prehm, Fred Weintraub 
Council on Exceptional Children (CEC) 

Bill Schipper 

National Association of State Directors 
of Special Education (MASDSB) 

Lou Bowers 

President, Rational Consortium on 

Physical Education for the Handicapped 



Phil Burke 

Chairman, Department of Special 

Education 
University of Maryland 
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Appendix F 

BIBLIOGRAPHY OF PROGRAM-RELATED DOCUMKHTS 



Studies and Reports: Personnel Preparat ion for Special Education 

Bale, R. L. , Grant, W. , & Hanley, T. V. (1982). The Statue of inservice 
training to prepare educators to work with han dicapped students (Executive 
Summary). Applied Management Sciences, Inc. 

Chalfant, J. C. , Duncan, J. R. , Meyen, B. L., Schofer, R. C. , & Ueberle, J. 
(1981). Comprehensive system of personnel devel opaent; Heads assessment 
considerations . Columbia, MO: University of Missouri. 

Darling-Hammond, L. , with Haggstrom, G. , Hudson, L, & Oakes, J. (1986). 
A conceptual framework for examining sta ffing and school in*. (Working 
draft intended to transmit preliminary results of Rand research to the 
Department of Education's Center for Statistics.) Santa Monica, CA: Rand. 

Duncan, J. R. , Schofer, R. C. , & Ueberle, J, (1982). comprehensive system of 
personnel development: Inservice considerations . Columbia, HO: 
University of Missouri. 

Frane, G. (1985). Special education personnel developm ent: Analysis of 
Arkansas/Texas study . Washington, DC: U.S. Department of Education, 
Office of Planning, Budget, and Evaluation. 

Geiger, W. L. (1983). 1983 national directory of special edu cation teacher 
education programs . Teacher Education Division of CBC. 

Hastings, A. H. , Hayward, B. J., & LeBlanc, L. A. (1984). The development of 
an information system to identify rehabilitation pers onnel shortages: 
Suggested options . Advanced Technology and Policy Studies Associates. 

Holland, R. P., & Noel, M. M. (1985). A review of federal legislation 
concerning special education personnel preparation . College Park, HD: 
Institute for the Study of Exceptional Children and Youth, Department of 
Special Education, University of Maryland. 

McLaughlin, M. J., Smith-Davis, J., & Burke, P. J. (1986). Personnel to 
educate the handicapped in America: A status report . College Park, HD; 
Institute for the Study of Exceptional Children and Youth, Department of 
Special Education, University of Maryland. 

Hoel, M. M., Spence, K. L. , & Valdivieso, C. H. (1986). Issues and policy 
implications for pr ^flXffflff SB fl» training of special educators: A 
synthesis of four national studies. Paper presented at the annual meeting 
of the American Educational Research Association, San Francisco, 
April 16-20, 1986. 
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Boel, M. M. , Valdivieso, c. H. f & Fuller, B. C. (1985). Determinants of 

teacher preparation: A study of departments of special education. Collage 
Park, MD: institute for the Study of Exceptional Children and Youth, 
Department of Special Education, University of Maryland. 

Schofer, R. C, 6 Duncan, J. H. (1984). A national survey of comprehensive 
systems of personnel development: A fifth status study . Columbia, MO: 
University of Missouri. 

Schofer, R. C. et al. (1984). Comprehensive system of personnel develop- 
ment; Indicators of effectiveness . Columbia, MO: University of Missouri. 

Smith-Davis, J., Burke, P. J. . & Boel, M. (1983). Personnel to educate the 
handicapped in America: Supply and demand from a pr^r am m n n^ viewpoint 
(Executive Summary). College Park, MD: University of Maryland. 

Smith-Davis, J., Mors ink, c. , & Wheat ley, F. W. (eds.) (1984). Quality in 
personnel preparation for education of the handicapped: The b a s c line 
book . Vienna, VA: Dissemin/Action. 

Spence, K. L. , Moel, M. M. , & Boyer-Shick, K. (1985). Summary report of a 
1985 survey of special education students . College Park, MD: Institute 
for the Study of Exceptional Children and Youth, Department of St> -ial 
Education, University of Maryland. 

U.S. Department of Education, office of the Assistant Secretary for 
Educational Research and improvement, National Center for Education 
Statistics. (1984). "Graduates Bewly Qualified to Teach in Special 
Education Programs: The 1981 Survey of 1979-80 College Graduates.." 
Special Report . 

U.S. Department of Education, office of Special Education and Rehabilitative 
Services, Special Education programs (1985). Fund inn Recommendations . 
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